Information recording medium, information recording device and method, information reproduction device and method, information recording/reproduction device and method, recording or reproduction control computer program, and data structure containing control signal

ABSTRACT

An information record medium comprises video information for indicating a main-video, sub-video information for indicating a sub-video which can be displayed at least partially over the main-video, and sub-video control information including (i) a plurality of types of control information elements for display control of the sub-video information in various methods set in advance and (ii) type indicating information to indicate presence or absence, or effectiveness or ineffectiveness of the control information elements by each of the types.

TECHNICAL FIELD

The present invention relates to: an information record medium, such asa high density optical disc, capable of recording various informationsuch as main-video information, audio information, sub-videoinformation, reproduction control information, and so on, at highdensity; an apparatus for and a method of recording the information ontothe information record medium; an apparatus for and a method ofreproducing the information from the information record medium; anapparatus and a method capable of both recording and reproducing theinformation; a computer program for controlling the recording orreproduction; and a data structure including a control signal forcontrolling the reproduction.

BACKGROUND ART

DVDs become common as optical discs onto which various information suchas main-video information, audio information, sub-video information,reproduction control information and so on is recorded. According to aDVD standard, the main-video information (video data), the audioinformation (audio data) and the sub-video information (sub-picturedata) are packetized each with the reproduction control information(navigation data) and multi-recorded onto a disc in a program streamformat of MPEG 2 (Moving Picture Experts Group phase 2), which is a highperformance encoding technology. Among them, the main-video informationhas data, which is compressed in a MPEG video format (ISO 13818-2), forone stream in one program stream. On the other hand, the audioinformation is recorded in a plurality of formats (i.e. linear PCM,AC-3, MPEG audio and so on) and has data for up to 8 streams in oneprogram stream. The sub-video information is defined by a bitmap, and iscompressed and recorded in a run length method, and has data for up to32 streams in one program stream.

On the other hand, a transport stream of MPEG2 standard is coming to bestandardized, which is suitable for data transfer. According to thetransport stream format, a plurality of elementary streams istransferred at the same time. For example, a plurality of programs, suchas a plurality of satellite digital broadcasting channels in onesatellite radio wave, is transmitted at the same time in a TDM (TimeDivision Multiplex) scheme.

In DVD's of this type, sub-picture information may be recorded as thesub-video information, such as a caption for a movie, in associationwith video information as the main-video information. Moreover, forexample, it is possible to superimpose and display the pictures ofvarious buttons, such as a button for menu selection and a button foroperation execution, over the main-video.

DISCLOSURE OF INVENTION

Also in the information record medium of this type, there is apossibility of more complex and sophisticated content reproduction, suchas an interactive reproduction, in accordance with the progress in thehigh density technology of the information record medium. Accordingly,with regard to the sub-picture also, more complex and sophisticatedreproduction control is required, in accordance with the interactivereproduction. For example, a variety of various display control isrequired on the main-picture-video, such as the dynamic display of thesub-picture, the display with a shadow of the sub-picture, and thedisplay of various operation buttons which are highlight controlled bythe sub-picture.

However, in order to perform the various display control by using thesub-picture in the DVD video standard, it is necessary to correctivelyread various control information required for the display control, orselectively read necessary information out of the various controlinformation, into a player from the DVD. It takes time to read thecontrol information, in addition to the reading of the main-videoinformation and the sub-video information, which causes such a problemthat it is technically difficult to perform the various display control,quickly enough not to stop continuous reproduction. Moreover, forexample, in order to temporarily stop or completely stop one displaycontrol, such as shading or highlighting the sub-picture, it isnecessary to eliminate or invalidate the control information for that,on a player. As a result, it takes time for eliminating andinvalidating, and it causes an increase in the processing load on acontroller, so that there is also a technical problem of complicatedreproduction control as a whole.

The present invention has been accomplished in view of the aboveproblems for example. It is therefore an object of the present inventionto provide an information record medium, an information record apparatusand method, an information reproduction apparatus and method, aninformation record reproduction apparatus and method, a computer programfor a record or reproduction control, and a data structure including acontrol signal, allowing a complex and sophisticated reproduction of thesub-video information to be reproduced with the main-video information,including a reproduction of the sub-picture playing a part of thecomplex and sophisticated content reproduction such as an interactivereproduction with the aid of a DVD player or the like.

On the information record medium of the present invention, there arerecorded: video information to indicate a main-video; sub-videoinformation to indicate a sub-video displayable at least partially overthe main-video; and sub-video control information including (i) aplurality of types of control information elements for display controlof the sub-video information in various methods set in advance and (ii)type indicating information to indicate presence or absence, oreffectiveness or ineffectiveness of the control information elements byeach of the types.

According to the information record medium of the invention, when theinformation on the information record medium, such as a DVD, isreproduced, the sub-video information such as the sub-pictureinformation can be superimposed at least partially as a sub-frame, forexample, over the main-video such as the main-picture or the videodisplay screen, as it is or after any processing or treatment.Incidentally, the entire sub-video information may be displayed over themain-video, or a part of the sub-video information may be cut out as thesub-frame and displayed. Alternatively, one or a plurality of sub-framesas described above is cut out from the same sub-video information. Inthe case that a plurality of sub-frames is cut out, they may be cut outincluding any common part, or including no common part.

In reproducing the information on the information record medium, thesub-video information is display-controlled in the various methods, onthe basis of a control information element whose presence oreffectiveness is shown by the type indication information included inthe sub-video control information, out of the plurality of types ofcontrol information elements which are set in advance and which are alsoincluded in the sub-video control information stored in a SCP(Sub-picture Control Packet) or recorded as a SCP stream. Here, the“display control in the various methods” includes, a variety of variousdisplay controls, such as the dynamic display of the sub-picture, thedisplay with a shadow of the sub-picture, and the display of variousoperation buttons which are highlight-controlled by the sub-picture, onthe main-video, for example. Moreover, the type indication informationindicates whether or not each of the control information elements ispresent in the sub-video control information, or whether or not each ofthe control information elements is effective regardless of itspresence, by using a binary value of “1” or “0”, for example. Thus, inthe reproduction, if the display control of the sub-video is performedon the main-video, i.e., if the display control over the sub-videoinformation is performed, the type indication information is firstlyreferred to, by an information reproduction apparatus. Then, the controlinformation element whose presence or effectiveness is confirmed by thistype indication information, is read from the information record medium,and the display control over the sub-video information is performed inthe various methods, in accordance with the read control informationelement.

Consequently, it is possible to perform the display control over thesub-video information as the sub-picture, in the various methods,efficiently and quickly, for example, and this eventually allows avariety of various displays of the sub-picture.

In one aspect of the information record medium of the present invention,the sub-video control information further includes sub-frame rangeinformation to specify at least a partial area of the sub-video as asub-frame, the control information element comprises information forcontrolling the sub-video information by a unit of the sub-frame, andthe type indication information indicates the presence or absence, orthe effectiveness or ineffectiveness of the control information elementsby the unit of the sub-frame.

According to this aspect, at least a partial area of the sub-video isdesignated as a sub-frame, by the sub-frame range information, such ascoordinate information or the like to designate coordinates of opposingtwo or four corners of the rectangular area of the sub-frame. Althoughthe sub-frame may be a rectangular area within the sub-video, thesub-frame can take any arbitrary shape. The control information elementis provided with information for controlling the sub-video informationby the unit of the sub-frame, and the type indication informationindicates the presence or absence, or the effectiveness orineffectiveness of the control information elements by the unit of thesub-frame. Thus, if the display control over the sub-frame is performedin the reproduction, firstly, the type indication information isreferred to by the unit of the sub-frame, and secondly, the controlinformation element whose presence or effectiveness is confirmed by thisis read from the information recording medium by the unit of thesub-frame, and the display control over the sub-frame is performed inthe various methods.

In another aspect of the information record medium of the presentinvention, the type indication information is provided with tableinformation including one bit of information by each of the types.

According to this aspect, if the display control over the sub-videoinformation is performed in reproducing the information from theinformation record medium, firstly, the type indication information,which is provided with the table information including one bit ofinformation by each of the types, is firstly referred to. For example,the one bit of information indicates, by its value “1”, that thecorresponding control information element is present or effective, andby its value “0”, that the corresponding control information element isabsent or ineffective. Thus, on the information reproducing apparatus,it is possible to confirm the presence or absence, or the effectivenessor ineffectiveness of the control information elements, relativelyquickly and easily, and it is possible to effectively perform thedisplay control over the sub-video information in the various methods asa whole.

In this aspect, it may be constructed such that the table informationincludes extension bit information which is not associated with thetypes, and a part occupied by the control information elements in thesub-video control information is variable-length.

According to this aspect, in the case where the display control isperformed in a new method which has not been included in the tableinformation, the extension bit information is replaced by the typeindication information for the display control in the new method. Then,the control information elements for the display control in the newmethod are newly included into a part occupied by the variable-lengthsub-video information. By these, the display control in the new methodscan be introduced, relatively easily.

In another aspect of the information record medium of the presentinvention, the control information element includes dynamic controlinformation to display the sub-video information dynamically.

According to this aspect, the sub-video is displayed over themain-video, in such a format that the sub-video receives the dynamicdisplay control by the dynamic control information. For example, a partof the sub-frame is subjected to a special treatment or specialprocessing, such as a translation (parallel displacement), a scaling(scaling up and scaling down), a rotation and so on, so that the part ofthe sub-frame is displayed dynamically, relative to the main-video, overthe reproduction time.

In another aspect of the information record medium of the presentinvention, the control information element includes shading-displaycontrol information to selectively shade an opaque part out of thesub-video and display it over the main-video.

According to this aspect, the sub-video is displayed over the main-videoafter the opaque part out of the sub-video is selectively shaded. Forexample, the opaque part is selectively shaded, depending on an alphavalue which indicates the transparency of the sub-video part. Then, forexample, letters or figures included in a rectangular sub-frame areshaded and displayed as those standing out three-dimensionally on themain-video. Alternatively, the entire sub-frame or the outer frame ofthe sub-frame is shaded and displayed on the main-video.

In another aspect of the information record medium of the presentinvention, the control information element includes (i) buttoninformation which defines at least a part of the sub-video which is tobe displayed over the main-video, as a button video part which functionsas a button, and (ii) highlight information which defines how to performhighlight-display control over the button video part.

According to this aspect, at least a part of the sub-video information,including a plan view, a pattern, an icon, a picture, and the like thatindicate a button, is defined by the button information included in thecontrol information element, as the button video part. The “button”herein is a push button (i.e. a button which can be simply pressed), atoggle button (i.e. a button whose status is changed over alternately ineach operation), an exclusive button (i.e. a plurality of buttons, onlyone of which can be pressed), and the like, for example. The operationof such a button in the reproduction is performed by a remote operation,an audio input operation, a touch operation with respect to a screen, akeyboard operation, and the like. Moreover, on the basis of thehighlight information included in the control information element, it isdefined how to perform the highlight-display control over the buttonvideo part. “How to perform the highlight-display control” hereinindicates what kind of highlight display is performed in response to the“state of the button”, such as pressed, selected, not-pressed,not-selected, can be pressed or selected, cannot be pressed or selected,and the like. The specific highlight display control is performed bydisplaying the button to be highlight-displayed so as to stand out fromother buttons or other positions, such as changing brightness, changinga contrast, and back shading or reverse shading, in response to thestate of the button. As a result, out of the sub-video, the button videopart defined by the button information is displayed over the main-videoas the button, and the highlight display control over the button videopart is performed in accordance with the highlight information.

In another aspect of the information record medium according to thepresent invention, the main-video information, the sub-video informationand the sub-video control information are divided into predeterminedpacket units and multiplexed, and further streamed relatively into avideo stream composed of the divided main-video information, asub-picture stream composed of the divided sub-video information set anda control information stream composed of the divided sub-video controlinformation.

According to this aspect, in the case that each information is recordedafter it is packetized and streamed, similarly to the PS (ProgramStream) of the MPEG2 for example, the sub-video information such as thesub-picture information is recorded as a special SPD (Sub-Picture Data)stream, and the sub-video control information to perform the displaycontrol for the sub-video information is recorded as a special SCP(Sub-picture Control Packet) stream, which are different from the SPDstream. The main-video is also recorded as a special video stream.Therefore, for example, a display based on the sub-video informationrecorded as one stream in the PS can be display-controlled efficientlyin the various methods by the sub-video control information includingthe plurality of types of control information elements, recorded asanother stream. Additionally, the display control in the various methodscan be also achieved by using a plurality of sub-video controlinformation streams relative to the same sub-video information stream.For example, it is also possible to perform different display controlsby using the same sub-video information.

The information record apparatus of the present invention is providedwith: a first record device for recording video information to indicatea main-video; a second record device for recording sub-video informationto indicate a sub-video displayable at least partially over themain-video; and a third record device for recording sub-video controlinformation including (i) a plurality of types of control informationelements for display control of the sub-video information in variousmethods set in advance and (ii) type indicating information to indicatepresence or absence, or effectiveness or ineffectiveness of the controlinformation elements by each of the types.

According to the information record apparatus of the present invention,the first record device which is made of a controller, an encoder, a TSobject generator mentioned below, an optical pickup, a cutting device orthe like, records the video information to indicate the main-video, ontothe information record medium such as a DVD. The second record devicewhich is made of a controller, an encoder, an optical pickup, a cuttingdevice or the like, records the sub-video information onto theinformation record medium such as a DVD. The third record device whichis made of a controller, an encoder, an optical pickup, a cutting deviceor the like, records the sub-video control information including thecontrol information elements and the type indication information, ontothe information record medium such as a DVD.

Therefore, it is possible to record the information onto the informationrecord medium of the present invention mentioned above (includingvarious aspects thereof), relatively efficiently.

Incidentally, the information record apparatus according to the presentinvention may also take various aspects, correspondingly to variousaspects of the information record medium according to the presentinvention as mentioned above.

The information record method of the present invention is provided with:a first record process of recording video information to indicate amain-video; a second record process of recording sub-video informationto indicate a sub-video displayable at least partially over themain-video; and a third record process of recording sub-video controlinformation including (i) a plurality of types of control informationelements for display control of the sub-video information in variousmethods set in advance and (ii) type indicating information to indicatepresence or absence, or effectiveness or ineffectiveness of the controlinformation elements by each of the types.

According to the information record method of the invention, the firstrecord process is for recording the video information to indicate themain-video, the second record process is for recording the sub-videoinformation, and the third record process is for recording the sub-videocontrol information including the control information elements and thetype indication information, onto the information record medium such asa DVD, with the aid of the controller, the encoder, a TS objectgenerator as described later, the optical pickup, the cutting device andso on.

Therefore, it is possible to record the information onto the informationrecord medium of the present invention as mentioned above (includingvarious aspects thereof), relatively efficiently.

Incidentally, the information record method according to the presentinvention may also take various aspects, correspondingly to variousaspects of the information record medium according to the presentinvention.

The information reproduction apparatus according to the presentinvention is an information reproduction apparatus for reproducinginformation on the information record medium of the present inventiondescribed above (including its various aspects), the informationreproduction apparatus comprising: a reproduction device for reproducingthe video information, the sub-video information and the sub-videocontrol information; a display output device capable of displaying andoutputting the reproduced sub-video information over the reproducedvideo information; and a control device for controlling the displayoutput device to display-control the sub-video information in thevarious methods and display the sub-video information over themain-video, on the basis of the control information element whosepresence or effectiveness is shown by the type indication informationwhich is included in the reproduced sub-video control information.

According to the information reproduction apparatus of the presentinvention, the reproduction device which is made of a controller, adecoder, a demultiplexer, an optical pickup and so on, reproduces thevideo information, the sub-video information and the sub-video controlinformation. The display output device, such as a CRT (Cathode Ray Tube)device, a PDP (Plasma Display Panel) device, an LCD (Liquid CrystalDisplay) device, a projector device and so on, can display and outputthe sub-video information over the reproduced video information. Then,the control device, such as a controller and the like, controls thedisplay output device to display-control the sub-video information inthe various methods and display the sub-video information over themain-video, on the basis of the control information element whosepresence or effectiveness is shown by the type indication informationwhich is included in the reproduced sub-video control information.

Therefore, it is possible to reproduce the information on theinformation record medium according to the present invention asmentioned above (including various aspects thereof), relativelyefficiently.

Incidentally, the information reproduction apparatus according to thepresent invention may also take various aspects, correspondingly tovarious aspects of the information record medium according to thepresent invention.

The information reproduction method according to the present inventionis an information reproduction method of reproducing information on theinformation record medium of the present invention described above(including its various aspects), the information reproduction methodcomprising: a reproduction process of reproducing the video information,the sub-video information and the sub-video control information; and acontrol process of controlling a display output device capable ofdisplaying and outputting the reproduced sub-video information over thereproduced video information, to display-control the sub-videoinformation in the various methods and display the sub-video informationover the main-video, on the basis of the control information elementwhose presence or effectiveness is shown by the type indicationinformation which is included in the reproduced sub-video controlinformation.

According to the information reproduction method of the presentinvention, the reproduction process is for reproducing the videoinformation, the sub-video information and the sub-video controlinformation, with the aid of the controller, the decoder, thedemultiplexer, the optical pickup and so on. The control process is forcontrolling the display output device such as the CRT device, the PDPdevice, the LCD device, the projector device and so on, with the aid ofthe controller and the like, to display-control the sub-videoinformation in the various methods and display the sub-video informationover the main-video, on the basis of the control information elementwhose presence or effectiveness is shown by the type indicationinformation which is included in the reproduced sub-video controlinformation.

Therefore, it is possible to reproduce the information on theinformation record medium of the invention mentioned above (includingvarious aspects thereof), relatively efficiently.

Incidentally, the information reproduction method according to thepresent invention may also take various aspects, correspondingly tovarious aspects of the information record medium according to thepresent invention as mentioned above.

The information record reproduction apparatus according to the presentinvention is provided with: a first record device for recording videoinformation to indicate a main-video; a second record device forrecording sub-video information to indicate a sub-video displayable atleast partially over the main-video; a third record process of recordingsub-video control information including (i) a plurality of types ofcontrol information elements for display control of the sub-videoinformation in various methods set in advance and (ii) type indicatinginformation to indicate presence or absence, or effectiveness orineffectiveness of the control information elements by each of thetypes; a reproduction device for reproducing the video information, thesub-video information and the sub-video control information; a displayoutput device capable of displaying and outputting the reproducedsub-video information over the reproduced video information; and acontrol device for controlling the display output device, todisplay-control the sub-video information in the various methods anddisplay the sub-video information over the main-video, on the basis ofthe control information element whose presence or effectiveness is shownby the type indication information which is included in the reproducedsub-video control information.

According to the information record reproduction apparatus of thepresent invention, since it has both functions of the information recordapparatus and the information reproduction apparatus of the inventionmentioned above, it is possible to record and reproduce the informationon the information record medium of the invention (including variousaspects thereof), relatively efficiently.

Incidentally, the information record reproduction apparatus according tothe present invention may also take various aspects, correspondingly tovarious aspects of the information record medium according to thepresent invention.

The information record reproduction method according to the presentinvention is provided with: a first record process of recording videoinformation to indicate a main-video; a second record process ofrecording sub-video information to indicate a sub-video displayable atleast partially over the main-video; a third record process of recordingsub-video control information including (i) a plurality of types ofcontrol information elements for display control of the sub-videoinformation in various methods set in advance and (ii) type indicatinginformation to indicate presence or absence, or effectiveness orineffectiveness of the control information elements by each of thetypes; a reproduction process of reproducing the video information, thesub-video information and the sub-video control information; and acontrol process of controlling a display output device capable ofdisplaying and outputting the reproduced sub-video information over thereproduced video information, to display-control the sub-videoinformation in the various methods and display the sub-video informationover the main-video, on the basis of the control information elementwhose presence or effectiveness is shown by the type indicationinformation which is included in the reproduced sub-video controlinformation.

According to the information record reproduction method of theinvention, since it has both functions of the information record methodand the information reproduction method of the invention mentionedabove, it is possible to record and reproduce the information on theinformation record medium of the invention (including various aspectsthereof), relatively efficiently.

Incidentally, the information record reproduction method according tothe present invention may also take various aspects, correspondingly tovarious aspects of the information record medium according to thepresent invention.

The computer program for a record control according to the presentinvention is to control a computer disposed at the information recordapparatus according to the present invention mentioned above (includingvarious aspects thereof), the program making the computer function as atleast a part of the first record device, the second record device andthe third record device.

According to the computer program for a record control of the presentinvention, the information record apparatus according to the presentinvention mentioned above may be embodied relatively easily, by readingand running the computer program from a record medium, such as a ROM, aCD-ROM, a DVD-ROM, a hard disk and so on, for storing the computerprogram therein/thereon, or by downloading the computer program to thecomputer via the communication device and running it.

Incidentally, the computer program for a record control according to thepresent invention may also take various aspects, correspondingly tovarious aspects of the information record medium according to thepresent invention as mentioned above.

The computer program for a reproduction control according to the presentinvention is to control a computer disposed at the informationreproduction apparatus according to the present invention mentionedabove (including various aspects thereof), the program making thecomputer function as at least a part of the reproduction device, thedisplay output device and the control device.

According to the computer program for a reproduction control of thepresent invention, the information reproduction apparatus according tothe present invention mentioned above may be embodied relatively easily,by reading and running the computer program from a record medium, suchas a ROM, a CD-ROM, a DVD-ROM, a hard disk and so on, for storing thecomputer program therein/thereon, or by downloading the computer programto the computer via the communication device and running it.

Incidentally, the computer program for a reproduction control accordingto the present invention may also take various aspects, correspondinglyto various aspects of the information record medium according to thepresent invention as mentioned above.

The computer program for a record reproduction control according to thepresent invention is to control a computer disposed at the informationrecord reproduction apparatus according to the present inventionmentioned above (including various aspects thereof), the program makingthe computer function as at least a part of the first record device, thesecond record device, the third record device, the reproduction device,the display output device and the control device.

According to the computer program for a record reproduction control ofthe present invention, the information record reproduction apparatusaccording to the present invention mentioned above may be embodiedrelatively easily, by reading and running the computer program from arecord medium, such as a ROM, a CD-ROM, a DVD-ROM, a hard disk and soon, storing the computer program therein/thereon, or by downloading thecomputer program to the computer via the communication device andrunning it.

Incidentally, the computer program for record reproduction controlaccording to the present invention may also take various aspects,correspondingly to various aspects of the information record mediumaccording to the present invention as mentioned above.

The data structure including a control signal according to the presentinvention has: video information to indicate a main-video; sub-videoinformation to indicate a sub-video displayable at least partially overthe main-video; and sub-video control information including (i) aplurality of types of control information elements for display controlof the sub-video information in various methods set in advance and (ii)type indicating information to indicate presence or absence, oreffectiveness or ineffectiveness of the control information elements byeach of the types.

According to the data structure including the control signal of thepresent invention, similarly to the information record medium accordingto the present invention mentioned above, it is possible to perform thedisplay control over the sub-video information as being the sub-picture,for example, in the various methods, efficiently and quickly, and thiseventually allows a variety of various displays of the sub-video on themain-video.

Incidentally, the data structure including the control signal accordingto the present invention may also take various aspects, correspondinglyto various aspects of the information record medium according to thepresent invention as mentioned above.

The above object of the present invention is achieved by a computerprogram product for a record control in a computer-readable medium fortangibly embodying a program of instructions executable by a computerdisposed at the aforementioned information record apparatus according tothe present invention (including its various aspects), the programmaking the computer function as at least a part of the first recorddevice, the second record device and the third record device.

The above object of the present invention is achieved by a computerprogram product for a reproduction control in a computer-readable mediumfor tangibly embodying a program of instructions executable by acomputer disposed at the information reproduction apparatus according tothe present invention (including its various aspects), the programmaking the computer function as at least a part of the reproductiondevice, the display output device and the control device.

The above object of the present invention is achieved by a computerprogram product for a record reproduction control in a computer-readablemedium for tangibly embodying a program of instructions executable by acomputer disposed at the information record reproduction apparatusaccording to the present invention (including its various aspects), theprogram making the computer function as at least a part of the firstrecord device, the second record device, the third record device, thereproduction device, the display output device and the control device.

According to the computer program product for the record control, thereproduction control, or the record reproduction control of theinvention, at least a part of the first record device, the second recorddevice, the third record device, the reproduction device, the displayoutput device and the control device according to the present inventionmentioned above may be embodied relatively easily, by reading andrunning the computer program product from a record medium, such as aROM, a CD-ROM, a DVD-ROM, a hard disk and so on, for storing thecomputer program therein/thereon, or by downloading the computer programproduct to the computer via the communication device and running it.More specifically, the computer program product may be made of computerreadable codes (or computer readable commands) to make the computerfunction as at least a part of the first record device, the secondrecord device, the third record device, the reproduction device, thedisplay output device and the control device.

These effects and other advantages of the present invention become moreapparent from the following embodiments and examples.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 illustrates, in its upper part, a general plan view of an opticaldisc as an embodiment of the information record medium of the presentinvention; and illustrates, in its lower part, a schematic conceptualdiagram of an area structure in a radius direction corresponding to thegeneral plan view in the upper part.

FIG. 2 illustrates a schematic conceptual diagram (FIG. 2(a)) of aconventional program stream of MPEG2; a schematic conceptual diagram(FIG. 2(b)) of a transport stream of MPEG2 used in the embodiment; and aschematic conceptual diagram (FIG. 2(c)) of a program stream of MPEG2used in the embodiment.

FIG. 3 is a diagram schematically illustrating a data structure recordedonto the optical disc in the embodiment.

FIG. 4 is a conceptual diagram hierarchically illustrating a detail of adata structure in each title shown in FIG. 3.

FIG. 5 is a conceptual diagram hierarchically illustrating a detail of adata structure in each playlist set shown in FIG. 3

FIG. 6 is a conceptual diagram schematically illustrating a detail of adata structure in each playlist set shown in FIG. 3.

FIG. 7 is a conceptual diagram schematically illustrating a detail of adata structure of each item shown in FIG. 6.

FIG. 8 is a conceptual diagram schematically illustrating a logicstructure of data in each title element shown in FIG. 4.

FIG. 9 is a conceptual view schematically illustrating a logic structureof data in each title element shown in FIG. 4, in a case that eachplaylist set is composed of one playlist.

FIG. 10 is a conceptual view schematically illustrating a detail of adata structure in each object shown in FIG. 3.

FIG. 11 is a view schematically illustrating a situation that anelementary stream for a program #1, shown in the upper column, and enelementary stream for a program #2, shown in the middle column, aremultiplexed to form a transport stream for these two programs, on thebasis of a time scale in a horizontal direction.

FIG. 12 is a conceptual view conceptually illustrating an image of TSpackets multiplexed in one transport stream in the embodiment, as apacket arrangement based on the time scale.

FIG. 13 is a view schematically illustrating a logic structure of dataon an optical disc in the embodiment, focusing on development from alogic hierarchy to an object hierarchy or an entity hierarchy.

FIG. 14 is a block diagram schematically illustrating an informationrecord reproduction apparatus in the embodiment.

FIG. 15 is a flow chart indicating a recording operation (part 1) of theinformation record reproduction apparatus in the embodiment.

FIG. 16 is a flow chart indicating a recording operation (part 2) of theinformation record reproduction apparatus in the embodiment.

FIG. 17 is a flow chart indicating a recording operation (part 3) of theinformation record reproduction apparatus in the embodiment.

FIG. 18 is a flow chart indicating a recording operation (part 4) of theinformation record reproduction apparatus in the embodiment.

FIG. 19 is a flow chart indicating a reproduction operation of theinformation record reproduction apparatus in the embodiment.

FIG. 20 is a conceptual diagram showing a specific example of the datastructure (FIG. 20(a)) of the SP control information to control thesub-picture data, and the SP data structure (FIG. 20(b)) including theSP data that is still picture data constituting the entity of thesub-picture data.

FIG. 21 is a conceptual view showing three types of sub-picturestructures, constructed from the SP control information and the SP datastructure shown in FIG. 20.

FIG. 22 is a schematic diagram showing a relationship among a SPD streamand a plurality of SCP streams, along a reproduction time scale.

FIG. 23 is a set of tables indicating a structure of the sub-picturecontrol packet in the embodiment.

FIG. 24 is a set of tables indicating the structure of the sub-picturecontrol packet in the embodiment, continued from FIG. 23.

FIG. 25 is a set of tables indicating the structure of the sub-picturecontrol packet in the embodiment, continued from FIG. 24.

FIG. 26 is a set of tables indicating the structure of the sub-picturecontrol packet in the embodiment, continued from FIG. 25.

FIG. 27 is another set of tables indicating the structure of thesub-picture control packet in the embodiment, continued from FIG. 25.

FIG. 28 is a conceptual diagram showing a relationship between thesub-picture and the sub-frame in the embodiment.

FIG. 29 is a conceptual diagram showing a designation mode to designatethe sub-frame from the sub-picture in the embodiment.

FIG. 30 is a conceptual diagram showing a display position of thesub-frame in the embodiment.

FIG. 31 is a conceptual diagram showing sub-frames in each sub-framecoordinate systems, and layout of main-video that is made of thesesub-frames in the main-video coordinate system.

FIG. 32 is a conceptual diagram showing a display mode with atranslation of the sub-frame in the main-video coordinate system in theembodiment.

FIG. 33 is a conceptual diagram showing a display mode with atranslation, a size change of the sub-frame in the main-video coordinatesystem in the embodiment.

FIG. 34 is a conceptual diagram showing a display mode with a rotationof the sub-frame in the main-video coordinate system in the embodiment.

FIG. 35 is a conceptual diagram showing a sub-frame disposed in themain-video coordinate system and a display mode with a display window inthe embodiment.

FIG. 36 is a conceptual diagram showing a display mode with a rotationand a size change of the sub-frame in the main-video coordinate systemin the embodiment.

FIG. 37 is a conceptual diagram showing a display mode with a movementof a sub-frame in a sub-picture in the embodiment.

FIG. 38 is a conceptual diagram showing a display mode of the sub-framein the main-video coordinate system, along with a moving path in theembodiment.

FIG. 39 is a flow chart showing an operational flow of obtaining drawingcontrol parameters in the embodiment.

FIG. 40 is a flow chart showing an operational flow of checking theSFCCI and obtaining the SFCC in the embodiment.

FIG. 41 is a flow chart showing an operational flow of a drawingprocessing in a sub-frame in the embodiment.

FIG. 42 is a flow chart showing an operational flow of a drawingprocessing in a sub-frame in the embodiment, continued from FIG. 41.

FIG. 43 is a flow chart showing an operational flow of a drawingprocessing in a sub-frame in the embodiment, continued from FIG. 42.

FIG. 44 is a flow chart showing an operational flow of a drawingprocessing in a sub-frame in the embodiment, continued from FIG. 43.

FIG. 45 is a conceptual illustration showing a shadow dropping displayin the embodiment.

FIG. 46 is a conceptual view showing a button display in the embodiment.

FIG. 47 is a illustration showing various button statuses in theembodiment.

FIG. 48 is a view conceptually illustrating a general flow of an accessduring a reproduction in the embodiment, showing in association with alogic structure of an optical disc.

FIG. 49 is a diagram schematically showing a specific example of a datastructure in an AU table constructed in the object information file andan ES map table associated with the AU table, in a specific example ofthe embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

(Information Record Medium)

The information record medium of the present invention is discussed,with reference to its embodiments, as well as FIG. 1 to FIG. 13. Inthese embodiments, the information record medium of the presentinvention is applied to an optical disc capable of recording (writing)and reproducing (reading).

Firstly, with reference to FIG. 1, a fundamental structure of theoptical disc in an embodiment is discussed. FIG. 1 illustrates, in itsupper part, a general plan view of the optical disc structure having aplurality of areas, and illustrates conceptually, in its lower part, anarea structure in the radius direction corresponding to the upper part.

As shown in FIG. 1, the optical disc 100 may be recorded by variousrecord methods, such as a magneto-optical method, a phase change method,capable of recording (writing) only once or a plurality of times.Similarly to DVDs, the optical disc 100 has a lead-in area 104, a dataarea 106 and a lead-out area 108, from the inner circumference around acenter hole 102 to the outer circumference, on the record surface of thedisc body measuring about 12 cm in diameter. In each area, groove tracksand land tracks may be alternately arranged, concentrically or spirallyaround the center hole 102. The groove tracks may be wobbled.Furthermore, pre-pits may be formed on one or both of these tracks.Incidentally, the present invention is not exclusively limited to theoptical disc having three areas mentioned above.

Next, with reference to FIG. 2, the structures of the transport stream(TS) and the program stream (PS) to be recorded onto the optical disc inthe embodiment are discussed. FIG. 2(a) schematically illustrates aMPEG2 program stream of a conventional DVD for a comparison, FIG. 2(b)schematically illustrates a MPEG2 transport stream (TS) structure.Furthermore, FIG. 2(c) schematically illustrates a MPEG2 program streamstructure in the present invention.

In FIG. 2(a), one program stream to be recorded in the conventional DVDincludes only one video stream for video data as main-pictureinformation, along the time axis t, and further includes up to 8 audiostreams for audio data as audio information, up to 32 sub-picturestreams for sub-picture data as sub-picture information. That is, thevideo data to be multiplexed at an arbitrary time point tx relates toonly one video stream. For example, a plurality of video streamcorresponding to a plurality of TV programs or a plurality of movies cannot be included at the same time in the program stream. It is notpossible to multiplex a plurality of TV programs and transfer or recordthem, in a program stream format of a DVD having only one video stream,because at least one video stream is required for each TV program, inorder to transfer or record the multiplexed TV program or the likeinvolving a video image.

In FIG. 2(b), one transport stream (TS) to be recorded in the opticaldisc 100 of the present invention includes a plurality of video streamsas elementary streams (ES) for video data as main-picture information,and further includes a plurality of audio streams as elementary streams(ES) for audio data as audio information and a plurality of sub-picturestreams as elementary streams (ES) for sub-picture as sub-pictureinformation. That is, the video data to be multiplexed at an arbitrarytime point tx relates to a plurality of video streams. For example, aplurality of video streams that may correspond to a plurality of TVprograms or a plurality of movies can be included at the same time inthe transport stream. Thus, it is possible to multiplex a plurality ofTV programs and transfer or record them, in the transport stream formathaving a plurality of video streams. However, the sub-picture stream isnot transferred in a digital broadcasting employing the existingtransport stream.

In FIG. 2(c), one program stream (PS) to be recorded onto the opticaldisc 100 of the present invention includes a plurality of video streamsfor video data as main-picture information, and further includes aplurality of audio streams for audio data as audio information and aplurality of sub-picture streams for sub-picture data as sub-pictureinformation. That is, the video data to be multiplexed at an arbitrarytime point tx relates to a plurality of video streams. For example, aplurality of video streams that may correspond to a plurality of TVprograms or a plurality of movies can be included at the same time inthe program stream.

Incidentally, for convenience of explanation, the video stream, theaudio stream and the sub-picture stream are arranged in this order fromthe top in FIG. 2(a) to FIG. 2(c). Nevertheless, this order or sequencedoes not correspond to an order or sequence for multiplexing packet bypacket as mentioned below. In the transport stream, conceptually, a setof one video stream, two audio streams and two sub-picture streamscorresponds to one program for example.

The optical disc 100 in the aforementioned embodiment is adapted tomulti-record the transport stream (TS) as shown in FIG. 2(b), i.e. torecord a plurality of programs at the same time. Furthermore, instead ofor in addition to this transport stream, the program stream (PS) asshown in FIG. 2(c) can be multi-recorded onto the same optical disc 100.

Next, with reference to FIG. 3 and FIG. 10, a structure of data to berecorded onto the optical disc 100 is discussed. FIG. 3 schematicallyillustrates the data structure to be recorded onto the optical disc 100.FIG. 4 schematically illustrates in detail the data structure in eachobject shown in FIG. 3. FIG. 5 and FIG. 6 schematically show a datastructure in detail respectively in each playlist (P list) set shown inFIG. 3. FIG. 7 schematically shows a detail of the data structure ofeach item shown in FIG. 6. FIG. 8 schematically shows a logic structureof data in each title element shown in FIG. 4. FIG. 9 schematicallyshows a logic structure of data in each title element, in a case thateach playlist set is composed of one playlist. FIG. 10 schematicallyshows a detail of a data structure in each object shown in FIG. 3.

In the following explanation, the “title” means a reproduction unit, onthe basis of which a plurality of “playlists” is executed continuouslyor sequentially, and which is a logically large grouped unit, such asone movie or one TV program. The “playlist set” means a bundle of“playlists”. For example, it may be a bundle of playlists to reproduce aplurality of contents information having a special relationshipswitchable to each other in an angle reproduction or a parentalreproduction, or may be a bundle of playlists to reproduce contentsinformation relating to a plurality of programs broadcasted in the sametime zone and collectively recorded. Alternatively, it may be a bundleof playlists to reproduce various contents information, in one title,prepared on the basis of required function, for example on the basis ofvideo performance or audio performance required for the informationreproduction system, such as a high vision compatibility, a displayresolution, a surround speaker compatibility, a speaker layout and soon. The “playlist” is information for storing the information requiredto reproduce the “object” and consists of a plurality of “items” eachstoring the information about a reproduction range of the object toaccess the object. The “object” is the entity information of contentsconstructing the aforementioned MPEG2 transport stream.

In FIG. 3, the optical disc 100 is provided with four files as a logicalstructure: a disc information file 110; a playlist (P list) informationfile 120; an object information file 130; and an object data file 140.The disc 100 is further provided with a file system 105 for managingthese files. Incidentally, although FIG. 3 does not show directly thephysical data arrangement on the optical disc 100, it is possible toperform the recording in such a manner that the arrangement shown inFIG. 3 corresponds to another arrangement shown in FIG. 1. That is, itis possible to record the file system 105 or the like in the data recordarea 106 following the lead-in area 104 and further record the objectdata file 140 or the like in the data record area 106. The filestructure shown in FIG. 3 can be constructed, even without the lead-inarea 104 or the lead-out area 108 shown in FIG. 1.

The disc information file 110 is a file for storing general informationabout the entire optical disc 100, and stores the disc generalinformation 112, the title information table 114 and other information118. The disc general information 112 may store the total numbers oftitles or the like in the optical disc 100. The title information table114 includes a title pointer 114-1 and a plurality of titles 200 (title#1-#m) whose ID (identification) number or record address is indicatedby the title pointer 114-1. In each title 200, each title type (e.g.sequential reproduction type, branch type and so on), or the playlist (Plist) number constructing each title is recorded for each title, as thelogical information.

More specifically, for example, as shown in FIG. 4, each title 200 ismade of a title general information 200-1, a plurality of title elements200-2 and other information 200-5. Furthermore, each title element 200-2is made of a pre-command 200PR, a pointer 20OPT to a playlist set, apost command 200PS and other information 200-6.

The pointer 200PT as an example of the first pointer informationaccording to the present invention indicates an ID number of theplaylist set 126S which is stored in the playlist information file 120and corresponds to contents information to be reproduced on the basis ofthe title element 200-2 including the pointer 200PT. Incidentally, thepointer 200PT may be information to indicate a record position of theplaylist set 126S corresponding to the contents information to bereproduced on the basis of the title element 200-2. The pre-command200PR as an example of the first pre-command according to the presentinvention indicates a command to be executed before reproducing thecontents information whose reproduction sequence is defined by oneplaylist set 126S designated by the pointer 200PT. The post command200PS as an example of the first post command according to the presentinvention is a command to be executed after reproducing the contentsinformation whose reproduction sequence is defined by said one playlistset. Other information 200-5 included in the title element 200-2 mayinclude next information to designate a title element for a nextreproduction after the present reproduction of the present titleelement, for example.

Therefore, when the information reproduction apparatus reproduces theinformation record medium, the desired contents information can bereproduced as a title element 200-2, by making access to the playlistset 126S in accordance with the pointer 200PT and performing a controlto select playlist corresponding to the desired program or the like fromamong a plurality of playlists 126 in the playlist set 126S.Furthermore, reproducing such a title element 200-2 solely orsequentially makes it possible to reproduce one title 200. Furthermore,it is possible to execute commands to be executed before reproduction,in the contents information whose reproduction sequence is defined byone playlist set 126S designated by the pointer 200PT, according to thepre-command 200PR. Furthermore, it is possible to execute commands to beexecuted after the reproduction, in the contents information whosereproduction sequence is defined by one playlist set 126S designated bythe pointer 200PT, according to the post command 200PS. The post command200PS may be a command to branch the contents information, a command toselect a next title and so on. Additionally, it is possible to reproducea next title element 200-2 to be reproduced after the presentlyreproduced title element 200-2, according to the “next information”included in other information 200-5.

Again in FIG. 3, the playlist information file 120 stores the playlist(P list) information table 121 indicating the logical structure of eachplaylist. This table 121 is divided into the playlist (P list)management information 122, the playlist (P list) set pointer 124, aplurality of playlist (P list) sets 126S (P list set #1-#n), and otherinformation 128. In this playlist information table 121, the logicalinformation of each playlist set 126S is stored in the order of theplaylist set number. In other words, the order for storing the eachplaylist set 126S is the playlist set numbers. Furthermore, in theaforementioned title information table 114, the same playlist set 126Scan be referred from a plurality of titles 200. That is, the playlistset #p in the playlist information table 121 may be pointed on the titleinformation table 114, even in the case that title #q and title #r usethe same playlist set #p.

As shown in FIG. 5, the playlist set 126S includes playlist set generalinformation 126-1, a plurality of playlists 126 (playlist #1-#x), anitem definition table 126-3, and other information 126-4. Each playlist126 includes a plurality of playlist elements 126-2 (playlist element#1-#y), and other information 126-5. Furthermore, each playlist element126-2 includes a pre-command 126PR, a pointer 126PT to item, a postcommand 126PS and other information 126-6.

The pointer 126PT as an example of the second pointer informationaccording to the present invention indicates an item identificationnumber defined by the item definition table 126-3 corresponding to thecontents information to be reproduced on the basis of the playlistelement 126-2 including the pointer 126PT. Incidentally, the pointer126PT may be an item record position defined by the item definitiontable 126-3.

As shown in FIG. 6, with regard to the playlist set 126S, a plurality ofitems 204 is defined in the item definition table 126-3. They arecommonly belonged to a plurality of playlists 126. Furthermore, as theplaylist set general information 126-1, a name of each playlist 126included in the playlist set 126S, UI (user interface information) suchas a reproduction time, address information to each item definitiontable 126-3 and so on are recorded.

Again in FIG. 5, the pre-command 126PR as an example of the secondpre-command according to the present invention indicates a command to beexecuted before reproduction of one item 204 designated by the pointer126PT. The post command 126PS as an example of the second post commandaccording to the present invention indicates a command to be executedafter the reproduction of said one item 204. Other information 126-6included in the playlist element 126-2 may include the next informationto designate the playlist element 126-2 relating to the nextreproduction of the reproduction of the playlist element 126-2.

As shown in FIG. 7, the item 204 is a minimum unit to be displayed. Inthe item 204, “in-point information” to indicate a start address of anobject and “out-point information” to indicate an end address of theobject are recorded. Incidentally, each of these “in-point information”and “out-point information” may indicate the address directly orindirectly as a time period or time point on the reproduction timescale. In the figure, if a plurality of ES's (Elementary Streams) ismultiplexed for the object designated by “stream object #m”, designatingthe item 204 means designating a special ES or special combination ofES's.

As shown in FIG. 8, the title element 200-2 logically consists of thepre-command 200PR or 126PR, the playlist set 126S selected by thepointer 200PT, the post command 200PS or 126PS, and the next information200-6N. Therefore, a processing to select the playlist 126 from amongthe playlist set 126S is executed, according to any conditionreproducible in a system, such as video resolution.

As shown in FIG. 9, however, if the playlist set designated by thepointer 200PT consists of only one playlist, i.e. if the playlist set126S shown in FIG. 3 is replaced by a single playlist 126, the titleelement 200-2 logically consists of the pre-command 200PR or 126PR, theplaylist 126 to be reproduced during the reproduction operation, thepost command 200PS or 126PS, and the next information 200-6N. In thiscase, once the playlist set is designated for the reproduction, thesingle playlist 126 is reproduced, regardless of the conditionreproducible in the system.

Again in FIG. 3, in the object information file 130, the storageposition (i.e. the logical address of the reproduction object) in theobject data file 140 for each item constructed in each playlist 126,and/or various attribute information relating to the reproduction of theitem are stored. Particularly in this embodiment, the object informationfile 130 stores the AU table 131 including a plurality of AU (AssociateUnit) information 132I (AU #1-#q) as mentioned below, the ES (ElementaryStream) map table 134 and other information 135.

The object data file 140 stores a plurality of TS objects 142 for eachtransport stream (TS #1 object-TS #s object), i.e. entity data ofcontents to be actually reproduced.

Incidentally, four kinds of file discussed with reference to FIG. 3 canbe further divided into a plurality of files respectively to be stored.All these files may be managed by the file system 105. For example, theobject data file 140 can be divided into a plurality of files such asobject data file #1, object data file #2 and so on.

As shown in FIG. 10, the TS object 142 shown in FIG. 3, as a logicallyreproducible unit, may be divided into a plurality of aligned units 143each having 6 kB of data amount. The head of the aligned units 143 isaligned with the head of the TS object 142. Each aligned unit 143 isfurther divided into a plurality of source packets 144 each having 192Bof data amount. The source packet 144 is a physically reproducible unit,on the basis of which (i.e. packet by packet) at least the video data,the audio data and the sub-picture data from among the data on theoptical disc 100 are multiplexed, and other information may bemultiplexed in the same manner. Each source packet 144 includes thecontrol information 145 having 4 B of data amount to control thereproduction, such as the packet arrival time stamp or the likeindicating the reproduction start time point of the TS (TransportStream) packet on the reproduction time scale, and includes the TSpacket 146 having 188 B of data amount. The TS packet 146 (also referredto as “TS packet payload”) has a packet header 146 a at its headportion. The video data may be packetized as the “video packet”, theaudio data may be packetized as the “audio packet”, or the sub-picturedata may be packetized as the “sub-picture packet”, otherwise other datamay be packetized.

Next, with reference to FIG. 11 and FIG. 12, an explanation is made onthe multi-recording of the video data, the audio data, the sub-picturedata and the like in the transport stream format as shown in FIG. 2(b),by the TS packet 146 shown in FIG. 4. FIG. 11 conceptually illustratesthat the elementary stream (ES) for program #1 (PG1) in the upper stageand the elementary stream (ES) for program #2 (PG2) in the middle stageare multiplexed, and the transport stream (TS) for these two programs(PG1&2) is formed, under an assumption that a horizontal axis is definedas a time axis. FIG. 12 conceptually illustrates an image of the TSpackets multiplexed in one transport stream (TS), as a packet arrayalong the time axis.

As shown in FIG. 11, the elementary stream for program #1 (in the upperstage) may be formed by discretely arranging TS packets 146 obtained bypacketizing the video data for program #1, along the time axis. Theelementary stream for program #2 (in the middle stage) may be formed bydiscretely arranging TS packets obtained by packetizing the video datafor program #2, along the time axis. Then, these TS packets 146 aremultiplexed so that the transport stream (in the lower stage) for twoprograms is made. Incidentally, omitted for convenience of explanationin FIG. 11, the elementary stream made of TS packets obtained bypacketizing the audio data and/or the sub-picture stream made of TSpackets obtained by packetizing the sub-picture data may be multiplexedsimilarly as the elementary stream for program #1, as shown in FIG.2(b). In addition to this, the elementary stream made of TS packetsobtained by packetizing the audio data and the sub-picture stream madeof TS packets obtained by packetizing the sub-picture data may bemultiplexed similarly as the elementary stream for program #2.

As shown in FIG. 12, in this embodiment, a plurality of TS packets 146multiplexed as such forms one TS stream. Then, a plurality of TS packets146 in the multiplexed form is multi-recorded onto the optical disc 100,with the information such as the packet arrival time stamp 145 and thelike being added. Incidentally, in FIG. 12, the TS packet 146 consistingof the data forming program #i (i=1, 2, 3) is indicated by “element (i0j)”, wherein, j (j=1, 2, . . . ) is a sequential number for each streamcomposing the program. This (i0 j) is defined as a packet ID which is anidentification number of the TS packet 146 for each elementary stream.This packet ID is fixed at an inherent value for a plurality of TSpackets 146 to be multiplexed at the same time point, so that theplurality of TS packets 146 is distinguished from each other even ifthey are multiplexed at the same time point.

Furthermore, in FIG. 12, the PAT (Program Association Table) and the PMT(Program Map Table) are also packetized by the TS packet 146 unit andmultiplexed. The PAT among them stores a table indicating a plurality ofPMT packet ID's. Particularly, the PAT is defined by MPEG2 standard sothat (000) is given as a predetermined packet ID, as shown in FIG. 12.That is, from among a plurality of packets multiplexed at the same timepoint, the TS packet 146 obtained by packetizing the PAT is detected, asthe TS packet 146 whose packet ID is (000). The PMT stores a tableindicating the packet ID for each elementary stream forming each programin one or more programs. Any packet ID can be given to the PMT, theirpacket IDs are indicated by the PAT detectable with the packet ID (000)as mentioned above. Therefore, among a plurality of packets multiplexedat the same time point, the TS packets 146 obtained by packetizing thePMT (i.e. TS packets 146 to which packet IDs (100), (200) and (300) aregiven in FIG. 12) are detected on the basis of the PAT.

In the case that the transport stream as shown in FIG. 12 is transferreddigitally, the tuner refers to the PAT and the PMT constructed as suchand thereby extracts the multiplexed packets corresponding to thedesired elementary stream and decodes the extracted packets.

In this embodiment, these PAT and PMT are included as the TS packets 146to be stored in the TS object 142 shown in FIG. 10. That is, when thetransport stream as shown in FIG. 12 is transferred, the transferredstream can be directly recorded onto the optical disc 100, which is agreat advantage.

Furthermore in this embodiment, these PAT and PMT recorded as such arenot referred to when the optical disc 100 is reproduced. Instead,referring to the AU table 131 and the ES map table 134, shown in FIG. 3and mentioned in detail later, makes it possible to perform thereproduction effectively and apply to the complicated multi-visionreproduction or the like. For this, in this embodiment, a relationshipbetween packets and the elementary stream obtained by referring to thePAT and the PMT on decoding or recording for example is stored in theobject information file 130, in a form of AU table 131 and ES map table134, without packetizing or multiplexing.

Next, with reference to FIG. 13, the logical structure of data on theoptical disc 100 is discussed. FIG. 13 schematically illustrates thelogical structure of data on the optical disc 100, focusing on thedevelopment from the logic hierarchy to the object hierarchy or theentity hierarchy.

In FIG. 13, one or more titles 200 that are a logical large unit such asone movie or one TV program are recorded onto the optical disc 100. Eachtitle 200 includes one or more title elements 200-2. Each title element200-2 logically consists of a plurality of playlist sets 126S. In eachtitle element 200-2, the plurality of playlist sets 126S may have asequential structure or may have a branch structure.

Incidentally, in the case of a simple logical structure, one titleelement 200 consists of one playlist set 126S. Furthermore, one playlistset 126S consists of one playlist 126. On the other hand, it is possibleto refer to one playlist set 126S by a plurality of title elements 200-2or a plurality of titles 200.

Each playlist 126 is logically made of a plurality of items (play items)204. In each playlist 126, a plurality of items 204 may have thesequential structure or may have the branch structure. On the otherhand, it is possible to refer to one item 204 by a plurality ofplaylists 126. The aforementioned in-point information and out-pointinformation recorded on/in the item 204 logically designates thereproduction range of the TS object 142. Then, the object information130d of the logically designated reproduction range is referred to andthereby the reproduction range of the TS object 142 is physicallydesignated, via the file system finally. Here, the object information130d includes various information to reproduce the TS object 142, suchas the attribute information of the TS object 142, the ES addressinformation 134d and the like required for the data search in the TSobject 142 (Incidentally, the ES map table 134 shown in FIG. 3 includesa plurality of ES address information 134 d).

Then, when the information record and reproduce apparatus reproduces theTS object 142 as mentioned below, a physical address to be reproduced inthe TS object 142 is obtained from the item 204 and the objectinformation 130 d so that a desired elementary stream is reproduced.

Incidentally, the EP (Entry Pass) map including a plurality of ESaddress information 134 d, shown within the object information of FIG.13, herein indicates an object information table in which the AU table131 and the ES map table 134 are listed.

Thus, in this embodiment, the in-point information and out-pointinformation recorded on/in the item 204, as well as the ES addressinformation 134 d recorded in the ES map table 134 (see FIG. 3) of theobject information 130 d make it possible to perform the associationfrom the logic hierarchy to the object hierarchy in the reproductionsequence so that the elementary stream is reproduced.

As discussed above, in this embodiment, units of the TS packet 146 aremultiplexed and recorded on the optical disc 100. Thereby, the transportstream including a plurality of elementary streams as shown in FIG. 2(b)can be recorded onto the optical disc 100. In this embodiment, in thecase that digital broadcasting is recorded onto the optical disc 100, aplurality of programs can be recorded at the same time, within therestriction of the record rate. Nevertheless, a record method isemployed herein, in which a plurality of programs is multiplexed andrecorded for one TS object 142. Now, an explanation is made on anembodiment of the information record reproduction apparatus capable ofperforming such a record processing.

(Information Record Reproduction Apparatus)

Next, with reference to FIG. 14 to FIG. 19, an embodiment of theinformation record reproduction apparatus of the present invention isdiscussed. Here, FIG. 14 is a block diagram of the information recordreproduction apparatus, and FIGS. 15 to 19 illustrate the operationalflow.

In FIG. 14, the information record reproduction apparatus 500 is roughlydivided into a reproduction system and a record system. The informationrecord reproduction apparatus 500 is constructed to record theinformation onto the optical disc 100 mentioned above and to reproducethe information recorded thereon/therein. In this embodiment, theinformation record reproduction apparatus 500 is thus for recording andreproduction. Nevertheless, an embodiment of the information recordapparatus according to the present invention can be constructedbasically with the record system of the information record reproductionapparatus 500 and an embodiment of the information reproductionapparatus according to the present invention can be constructedbasically with the reproduction system of the information recordreproduction apparatus 500.

The information record reproduction apparatus 500 is provided with: anoptical pickup 502; a servo unit 503; a spindle motor 504; a decoder506; a demultiplexer 508; a video decoder 511; an audio decoder 512; asub-picture decoder 513; an adder 514; a still picture decoder 515; asystem controller 520; a memory 530; a memory 540; a memory 550; amodulator 606; a formatter 608; a TS object generator 610; a videoencoder 611; an audio encoder 612; and a sub-picture encoder 613. Thesystem controller 520 includes a file system/logical structure datagenerator 521 and a file system/logical structure data reader 522.Furthermore, the memory 530 and a user interface 720 to give a userinput such as title information are connected to the system controller520.

Among these constitutional elements, the decoder 506, the demultiplexer508, the video decoder 511, the audio decoder 512, the sub-picturedecoder 513, the adder 514, the still picture decoder 515, the memory540 and the memory 550 mainly constructs the reproduction system. On theother hand, among these constitutional elements, the modulator 606, theformatter 608, the TS object generator 610, the video encoder 611, theaudio encoder 612 and the sub-picture encoder 613 mainly constructs therecord system. The optical pickup 502, the servo unit 503, the spindlemotor 504, the system controller 520, the memory 530 and the userinterface 720 to give the user input such as title information aregenerally shared for both the reproduction system and the record system.Furthermore, for the record system, a TS object data source 700 (or a PSobject data source 700, or a still picture data source 700 such as bitmap data, JPEG data and the like); a video data source 711; an audiodata source 712; and a sub-picture data source 713 are prepared.Furthermore, the file system/logical structure data generator 521disposed in the system controller 520 is used mainly in the recordsystem, and the file system/logical structure reader 522 is used mainlyin the reproduction system.

The optical pickup 502 irradiates the optical disc 100 with a light beamLB such as a laser beam, at the first power as reading light for thereproduction, and at the second power with the light beam LB beingmodified as writing light for recording. The servo unit 503 performs thefocus servo, the tracking servo and the like for the optical pickup 502,as well as the spindle servo for the spindle motor 504, under control ofthe control signal Sc1 outputted from the system controller 520, duringthe reproduction and recording. The spindle motor 504 is controlledunder the spindle servo by the servo unit 503, for rotating the opticaldisc 100 at a predetermined speed.

(i) Structure and Operation of Record System

Next, with reference to FIG. 14 to FIG. 18, a specific structure and theoperation of each constitutional element constructing the record systemof the information record and reproduction system 500 is explained ineach case.

(i-1) In the Case That the Already Generated TS Object is Used

This case is discussed, with reference to FIG. 14 and FIG. 15.

In FIG. 14, the TS object data source 700 may be made of the storagedevice such as a video tape, a memory, for storing the TS object dataD1.

Firstly in FIG. 15, each title information (e.g. playlist contents etc.)to be logically constructed on the optical disc 100 using the TS objectdata D1 is inputted into the system controller 520, as the user input I2such as the title information, via the user interface 720. Then, thesystem controller 520 imports the user input I2 such as the titleinformation via the user interface 720 (step S21: Yes, and step S22). Inthis case, the user interface 720, under control of the control signalSc4 from the system controller 520, can perform the input operation inresponse to the contents to be recorded, such as the selection via thetitle menu screen. Incidentally, if the user input is already performed(step S21: No), these processings are omitted.

Next, the TS object data source 700 outputs the TS object data D1, undercontrol of the control signal Sc8 to indicate the data reading from thesystem controller 520. Then, the system controller 520 imports the TSobject data D1 from the TS object data source 700 (step S23), andperforms the data array analysis (e.g. a record data length and thelike) of the TS object data D1, the analysis of each elementary streamstructure (e.g. understanding of ES_PID (elementary stream/packetidentification number)), on the basis of the PAT, the PMT and the likepacketized with the video data as mentioned above, due to the TSanalysis feature in the file system/logical structure data generator 521(step S24).

Next, the system controller 520 makes the file system/logical structuredata generator 521 generate the disc information file 110, the playlistinformation file 120, the object information file 130 and the filesystem 105 (see FIG. 3), as the logical information file data D4, on thebasis of the analysis result of each elementary stream and the TS objectdata D1 data array, as well as the user input I2 such as the importedtitle information (step S25). The memory 530 is used to generate thislogical information file data D4.

Incidentally, variations in which the data about structure informationof each elementary stream and the data array of the TS object data D1may be prepared in advance are naturally understood or suggested, all ofwhich are encompassed within a scope of the embodiment.

In FIG. 14, the formatter 608 is for formatting the data array to storeboth the TS object data D1 and the logical information file data D4 onthe optical disc 100. More specifically, the formatter 608 is providedwith a switch Sw1 and a switch Sw2 and is switching-controlled by aswitch control signal Sc5 from the system controller 520. Whenformatting the TS object data D1, it connects the switch Sw1 to a{circle around (1)} side and the switch Sw2 to a {circle around (1)}side so as to output the TS object data D1 from the TS object datasource 700. Incidentally, the transmission control of the TS object dataD1 is performed by the control signal Sc8 from the system controller520. On the other hand, when formatting the logical information filedata D4, the formatter 608 is switching-controlled by the switch controlsignal Sc5 from the system controller 520, and connects the switch Sw2to a {circle around (2)} side so as to output the logical informationfile data D4.

At step S26 in FIG. 15, (i) the logical information file data D4 fromthe file system/logical structure data generator 521 at the step S25 or(ii) the TS object data D1 from the TS object data source 700 isoutputted through the formatter 608, under the switching-control by theformatter 608 as constructed above (step S26).

The selected output from the formatter 608 is transmitted to themodulator 606 as disc image data D5, and modulated by the modulator 606,and recorded onto the optical disc 100 through the optical pickup 502(step S27). The system controller 520 also executes the disc recordcontrol in this case.

Then, if both the logical information file data D4 generated at the stepS25 and the corresponding TS object data D1 have not been completelyrecorded yet, the operational flow returns to the step S26 to continuethe recording (step S28: No). Incidentally, there is no preference inthe record order of the logical information file data D4 and thecorresponding object data D1.

On the other hand, if the both have been already recorded, it is judgedwhether or not the recording onto the optical disc 100 is to be ended,on the basis of the presence or absence of an end command (step S29). Ifit is not to be ended (step S29: No), the operational flow returns tothe step S21 to continue the recording. On the other hand, if it is tobe ended (step S29: Yes), a series of record processing ends.

As described above, the information record reproduction apparatus 500performs the record processing in the case of using the already preparedTS object.

Incidentally, the example in FIG. 15 shows that the logical informationfile data D4 and the corresponding TS object data D1 are outputted atthe step S26, after preparing the logical information file data D4 atthe step S25. However, it is also possible to output the TS object dataD1 and/or record the TS object data D1 onto the optical disc 100 beforethe step S25, so that the logical information file data D4 is generatedand/or recorded after or in parallel with this recording.

Additionally, a PS object data source or a still picture data source maybe used instead of the TS object data source 700. In this case, therecording processing the same as in the case of the TS object data D1 asmentioned above is performed for the PS object data, or the sill picturedata such as bit map data, JPEG data and the like, instead of the TSobject data D1. Furthermore, instead of the TS object 142, the PS objectdata or the still picture data is stored in the object data file 140.Then, various logical information about the PS object data or the stillpicture data is generated under control of the system controller 520,and stored in the disc information file 110, the playlist informationfile 120, the object information file 130 and the like.

(i-2) The Case of Receiving and Recording the Transport Stream on Air

This case is explained with reference to FIG. 14 and FIG. 16.Incidentally, in FIG. 16, the same steps as those in FIG. 15 have thesame step reference numbers, and their explanation is omitted asoccasion demands.

Again, in this case, the similar processing is performed, as is “thecase of using the already prepared object” described above. Therefore,the following explanation is focused on the differences from the abovecase.

In the case of receiving and recording the transport stream on air, theTS object data source 700 is provided with a receiver (set top box) forreceiving the digital broadcast on air, for example, receives the TSobject data D1, and transmits it to the formatter 608 in real time (stepS41). At the same time, reception information D3 (i.e. informationcorresponding to the data transmitted through the receiver and theinterface of the system controller 520) including the programconstruction information and the belowmentioned ES_PID information,which are deciphered upon receiving, is imported into the systemcontroller 520 and is stored into the memory 530 (step S44).

In the meantime, the TS object data D1 outputted to the formatter 608 isoutputted to the modulator 606 under the switching-control by theformatter 608 (step S42), and is recorded onto the optical disc 100(step S43).

Along with these operations, using the program construction informationand the ES_PID information included in the reception information D3imported upon receiving and stored in the memory 530, the filesystem/logical structure data generator 521 prepares the logicalinformation file data D4 (step S24 and step S25). Then, after thecompletion of recording a series of the TS object data D1, this logicalinformation file data D4 is additionally recorded onto the optical disc100 (step S46 and step S47). Incidentally, these steps S24 and S25 maybe performed after the step S43.

Moreover, as the occasion demands (e.g. in the case of editing oneportion of the title, or the like), by adding the user input I2 of thetitle information and the like from the user interface 720 to theprogram construction information and the ES_PID information stored inthe memory 530, it is possible to prepare the logical information filedata D4 by the system controller 520 and additionally record this ontothe optical disc 100.

As described above, the information record reproduction apparatus 500performs the record processing in the case of receiving the transportstream on air and recording it in real time.

Incidentally, if all the reception data obtained when broadcasting isonce stored into an archive apparatus, and then, if this is used as theobject source 700, the same processing as that in “the case of using thealready prepared TS object” will do.

(i-3) The Case of Recording the Video Data, the Audio Data and theSub-Picture Data

This case is explained with reference to FIG. 14 and FIG. 17.Incidentally, in FIG. 17, the same steps as those in FIG. 15 have thesame step reference numbers, and their explanation is omitted asoccasion demands.

In the case of recording the video data, the audio data, and thesub-picture data, which are individually prepared in advance, the videodata source 711, the audio data source 712, and the sub-picture datasource 713 are individually provided with the memory storage, such as avideo tape and a memory, and store a video data DV, an audio data DA,and a sub-picture data DS, respectively.

These data sources are controlled by the control signal Sc8 giving aninstruction for reading out the data from the system controller 520, andthey transmit the video data DV, the audio data DA, and the sub-picturedata DS, to the video encoder 611, the audio encoder 612, and thesub-picture encoder 613, respectively (step S61). Then, the videoencoder 611, the audio encoder 612, and the sub-picture encoder 613execute a predetermined type of encode processing (step S62).

The TS object generator 610 is controlled by a control signal Sc6 fromthe system controller 520 and converts the data encoded in this mannerto the TS object data constituting the transport stream (step S63). Inthis case, the data array information of each TS object data (e.g. arecord data length and the like) and the construction information ofeach elementary stream (e.g. the ES_PID, as described later, and thelike) are transmitted as information 16 from the TS object generator 610to the system controller 520 and are stored into the memory 530 (stepS66).

On the other hand, the TS object data generated by the TS objectgenerator 610 is transmitted to the {circle around (2)} side of theswitch Sw1 of the formatter 608. Namely, when formatting the TS objectdata from the TS object generator 610, the formatter 608 isswitching-controlled by the switch control signal Sc5 from the systemcontroller 520 to shift the switch Sw1 to the {circle around (2)} sideand the switch Sw2 to the {circle around (1)} side, thereby outputtingthe TS object data (step S64). Then, this TS object data is recordedonto the optical disc 100 through the modulator 606 (step S65).

Along with these operations, using the data array information of each TSobject data and the construction information of each elementary streamimported as the information 16 into the memory 530, the filesystem/logical structure data generator 521 prepares the logicalinformation file data D4 (step S24 and step S25). Then, after thecompletion of recording a series of the TS object data D2, the logicalinformation file data D4 is additionally recorded onto the optical disc100 (step S67 and step S68). Incidentally, the step S24 and the step S25may be processed after the step S65.

Moreover, as the occasion demands (e.g. in the case of editing oneportion of the title), by adding the user input I2 such as the titleinformation and the like from the user interface 720 to theseinformation stored in the memory 530, it is possible to prepare thelogical information file data D4 by the file system/logical structuregenerator 521 and additionally record this onto the optical disc 100.

As described above, the information record reproduction apparatus 500performs the record processing in the case of recording the video data,the audio data, and the sub-picture data, which are individuallyprepared in advance.

Incidentally, this record processing is applicable even when recordingan arbitrary content the user has.

(i-4) The Case of Recording the Data by Authoring

This case is explained with reference to FIG. 14 and FIG. 18.Incidentally, in FIG. 18, the same steps as those in FIG. 15 have thesame step reference numbers, and their explanation is omitted asoccasion demands.

In this case, by combining the above described three types of recordprocessing in the three cases, an authoring system generates the TSobject, the logical information file data, and the like in advance (stepS81), and then completes the processing until switching-controlperformed at the formatter 608 (step S82). Then, the informationobtained by this operation is transmitted, as the disc image data D5, tothe modulator 606 equipped in front of and/or behind an original disccutting machine (step S83), and this cutting machine prepares theoriginal disc (step S84).

(ii) Structure and Operation in Reproduction System

Next, the specific structure and operation of each constitutionalelement constituting the reproduction system of the information recordreproduction apparatus 500 is explained with reference to FIG. 14 andFIG. 19.

In FIG. 14, via the user interface 720, the title to be reproduced fromthe optical disc 100, its reproduction condition and the like areinputted to the system controller 520, as the user input I2 such as thetitle information and the like. In this case, under control of thecontrol signal Sc4 from the system controller 520, the input processingsuitable for the content to be reproduced, such as a selectionprocessing on a title menu screen, can be achieved by the user interface720.

Responding to this, the system controller 520 controls the discreproduction with respect to the optical disc 100, and the opticalpickup 502 transmits a reading signal S7 to the demodulator 506.

The demodulator 506 demodulates a recorded signal recorded onto theoptical disc 100 from this reading signal S7, and outputs it asdemodulated data D8. The logical information file data (i.e. the filesystem 105, the disc information file 110, the P list information file120, and the object information file 130, shown in FIG. 3) included inthis demodulated data D8 as being a not-multiplexed information part issupplied to the system controller 520. On the basis of this logicalinformation file data, the system controller 520 executes variousreproduction control, such as processing of determining a reproductionaddress and controlling the optical pickup 502.

On the other hand, depending on whether the TS object data is includedas the multiplexed information part in the demodulated data D8, orwhether the still picture data is included, or whether both data areincluded, the shift switch Sw3 is shifted to {circle around (1)} side(demultiplexer 508 side) or shifted to {circle around (2)} side (stillpicture decoder 515 side), under control of the control signal Sc10 fromthe system controller 520. Thereby, the TS object data is selectivelysupplied to the demultiplexer 508, and the still picture data isselectively supplied to the still picture decoder 515.

Then, as for the TS object data included as the multiplexed informationpart in the demodulated data D8, the demultiplexer 508 demultiplexes theTS object data, under control of the control signal Sc2 from the systemcontroller 520. Here, when the access to the reproduction positionaddress is terminated under the reproduction control by the systemcontroller 520, the control signal Sc2 is transmitted to start thedemultiplexing.

The video packet, the audio packet and the sub-picture packet aretransmitted respectively from the demultiplexer 508 and suppliedrespectively to the video decoder 511, the audio decoder 512 and thesub-picture decoder 513. Then, the video data DV, the audio data DA andthe sub-picture data DS are decoded, respectively. In this case, thesub-picture data DS is supplied to the adder 514 via the memory 540. Thesub-picture data DS is outputted from the memory 540, selectively or ata predetermined timing, under control of the control signal Sc5 from thesystem controller 520, so as to be super-imposed on the video data DV,if needed. That is, in comparison with a case that the sub-picture dataoutputted from the sub-picture decoder 513 is directly super-imposed, itis easy to control the timing of the super-imposing, or to judged theneed for the super-imposing. For example, under output control of thecontrol signal Sc5, it is possible to select whether or not a captionusing the sub-picture is displayed over the main-picture, or whether ornot a menu screen using the sub-picture is displayed.

Incidentally, although the packets obtained by packetizing the PAT orthe PMT, included in the transport stream shown in FIG. 6, are includedas a part of the demodulated data D8, respectively, they are discardedor abandoned at the demultiplexer 508.

The adder 514 is controlled by a control signal Sc3 giving aninstruction of the mixing from the system controller 520, and mixes orsuperimposes in a predetermined timing the video data DV and thesub-picture data DS, which are respectively decoded at the video decoder511 and the sub-picture decoder 513. The result is outputted as a videooutput from the information record reproduction apparatus 500 to a TVmonitor, for example.

On the other hand, the audio data DA decoded at the audio decoder 512 isoutputted as an audio output from the information record reproductionapparatus 500 to an external speaker, for example.

In the case that the still picture data is included in the decoded dataD8, the still picture data is supplied to the still picture decoder 515,via the shift switch Sw3 controlled by the control signal Sc10 from thesystem controller 520, instead of or in addition to such a reproductionoperation or processing of the video data DV or the sub-picture data DS.Then, the still picture data such as the decoded bit map data, JPEG dataand the like is directly added to the adder 514 via the shift switchSw4, under control of the control signal Sc11 from the system controller520. Alternatively, it may be temporarily stored in the memory 550 viathe shift switch Sw4. The still picture data is outputted from thememory 550, selectively or at a predetermined timing, under control ofthe control signal Sc12 from the system controller 520, and thensupplied to the adder 514 via the shift switch Sw5. Thereby, if needed,the still picture is super-imposed over the video data DV or thesub-picture data DS. That is, in comparison with a case that the stillpicture data outputted from the still picture decoder 515 is directlysuper-imposed, it is easy to control the timing of the super-imposing,or to judge the need for the super-imposing. For example, under outputcontrol of the control signal Sc12, it is possible to select whether ornot a still picture such as a menu screen or a window screen using thestill picture data or a still picture as a background image using thestill picture data is displayed over the main-picture or thesub-picture.

Additionally, under control of the control signal Sc13 from the systemcontroller 520, the still picture data may be outputted via anotherroute (not shown), with the aid of the shift switch Sw5 shifted to{circle around (2)} side. Alternatively, no still picture data may beoutputted from the shift switch Sw5 shifted to {circle around (2)} side.

Here, the specific example of a reproduction processing routine by thesystem controller 520 is explained with reference to a flow chart ofFIG. 19.

In FIG. 19, it is assumed that as an initial condition, the recognitionof the optical disc 100 in the reproduction system and the recognitionof a volume structure and a file structure by the file system 105 (seeFIG. 3) have been already completed by the system controller 520 and thefile system/logical structure data reader 522 inside of the systemcontroller 520. Here, an explanation is made on the operational flowafter obtaining the total number of the total titles from the discgeneral information 112 in the disc information file 110 and thenchoosing or selecting one title from among them.

Firstly, the choice or selection of the title is performed via the userinterface 720 (step S211). Responding to this, the system controller 520obtains the information about the reproduction sequence from a readingresult of the file system/logical structure data reader 522.Incidentally, in the selection of the title 200, the desired titleelement(s) 200-2 (see FIG. 4) may be selected from among a whole titleelements 200-2 composing the title 200, with the aid of an externalinput operation by the user with using a remote controller and the like,or one title element 200-2 may be selected automatically depending on asystem parameter or the like adjusted for the information recordreproduction apparatus 500.

Then, contents of a plurality of playlists 126 composing a playlist set126S corresponding to the selected title 200 (title element 200-2) areobtained. Here, as a processing at a logic hierarchy, the informationabout the each playlist 126 structure and each item composing eachplaylist (see FIG. 5, FIG. 6 and FIG. 13) is obtained (step S212).

Then, contents of the playlist 126 to be reproduced is obtained fromamong a plurality of playlists 126 obtained at step S212. For example,herein, the reproduction is started from a playlist #1, and the contentsof the corresponding playlist 126 is obtained (step S213). The contentsof the playlist 126 may be one or more playlist elements 126-2 (see FIG.5), which are obtained by the obtaining processing at step S213.

Then, the pre-command 126PR (see FIG. 5) included in this playlist 126is executed (step S214). Incidentally, it is possible for thepre-command 126PR to select one from among a plurality of playlists 126,which compose the playlist set 126S with a certain relationship of theplurality of playlists 126. If the playlist element 126-2 composing theplaylist 126 does not have the pre-command 126PR, this processing isomitted.

Then, the TS object 142 (see FIG. 3 and FIG. 10) to be reproduced isdetermined (step S215), on the basis of the item 204 (see FIG. 5 to FIG.7) identified by the playlist 126 obtained at step S213. Morespecifically, on the basis of the item 204, the object information file130 (see FIG. 3) relating to the TS object 142 as the reproductiontarget is obtained and a stream number, address and the like of the TSobject 142 to be reproduced are identified.

Incidentally, in this embodiment, also the belowmentioned AU(Association Unit) information 132I and PU (Presentation Unit)information 302I are obtained as the information stored in the objectinformation file 130. On the basis of these informations, theaforementioned logic hierarchy is associated with the object hierarchy(see FIG. 13).

Then, the reproduction of the TS object 142 determined at step S215 isactually started. That is, on the basis of the processing at the logichierarchy, the processing at the object hierarchy is started (stepS216).

During the reproduction of the TS object 142, it is judged whether ornot the next item 204 composing the playlist 126 to be reproduced exists(step S217). Then, insofar as the next item 204 exists (step S217: Yes),the process goes back to the step S215 to repeat the aforementioneddetermination and the reproduction of the TS object 142.

On the other hand, at the judgement at step S217, if it is judged thatthe next item 204 does not exist (step S217: No), the post command 126PS(see FIG. 5) corresponding to the presently executed playlist 126 isexecuted (step S218). Incidentally, if the playlist element 126-2composing the playlist 126 does not have the post command 126PS, thisprocessing is omitted.

Then, it is judged whether or not the next playlist 126 composing theselected title 200 exists (step S219). If exists (step S219: Yes), theprocess goes back to the step S213 to repeat the processing of obtainingthe playlist 126 to be reproduced and the subsequent processings.

On the other hand, at the judgement at step S219, if it is judged thatthe next playlist 126 does not exist (step S219: No), i.e. if the allplaylists 126 to be reproduced corresponding to the title 200 selectedat step S211 are completely reproduced, a series of reproductionoperations or processings is terminated.

As discussed above, the information record reproduction apparatus 500 inthis embodiment reproduces the optical disc 100.

Particularly in this embodiment, (i) with regard to the structure andoperation of the record system mentioned above, the object data file 140is recorded so that the sub-picture stream or the control informationstream, which are elementary streams relating to the sub-picture,includes the SP data (still picture data) and the SP control informationto control various displays of the sub-picture data.

Particularly in this embodiment, (ii) with regard to the structure andoperation of the reproduction system, when the object is determined andreproduced at steps S215 and S216, the SP data and the SP controlinformation recorded in the sub-picture stream or the controlinformation stream are reproduced, and thereby various display controlscan be achieved including the motion control, the shadow droppingcontrol, the highlight button control and so on of the sub-picture withusing the sub-frame. These various display controls of the sub-picturewill be discussed later in detail.

(Selection Scheme of Playlist in Playlist Set)

In this embodiment, the playlist 126 corresponding to the desiredcontents information is suitably selected from the playlist set 126Sincluded in the reproduced playlist information file 120.

Such a selection of the playlist may be performed in accordance with aselection condition, if the pre-command 200PR (see FIG. 4) included inthe title element 200-2 is provided with a list of playlist selectioncommands in which selection conditions are recorded for each playlist126. The selection of the playlist may be performed in accordance withthe attribute information added to each playlist 126 stored in theplaylist set 126S (e.g. the information to indicate the attribute of thecontents information relating to the playlist, including the videoresolution for the video performance, whether progressive or interlace,the video codec, the audio channel number, the audio codec and so on).Alternatively, the selection of the playlist may be performed inaccordance with the playlist set control information to store selectionconditions for each playlist. Due to these selections, for example, thedesired program, the desired parental block, the desired angle block andso on, which correspond to the desired contents information, can beselected. Alternatively, for example, a selection of the playlist can beachieved, so that the playlist displayable by the informationreproduction system is selected, and the video reproduction performanceor the audio reproduction performance of the information reproductionsystem is fully exerted.

(Various Display Controls of Sub-Picture)

Now, with reference to FIG. 20 to FIG. 47, an explanation is made onvarious display controls of the sub-picture with using the SP data andthe SP control information.

Firstly, with reference to FIG. 20 to FIG. 22, an explanation is madegenerously on the structure and control of the sub-picture data used toperform these various display controls of the sub-picture. From amongthem, the SP control information is discussed later in detail withreference to drawings from FIG. 23. FIG. 20(a) conceptually shows aspecific example of the data structure of the SP control information tocontrol the sub-picture data, and FIG. 20(b) conceptually shows aspecific example of the SP data structure including the SP data as thestill picture data consisting the main body of the sub-picture data.FIG. 21 shows schematically three types of sub-picture structurescomposed of the SP control information and the SP data structure. FIG.22 shows schematically a relationship between a SPD stream and aplurality of SCP streams along the reproduction time scale.

In this embodiment, in FIG. 14, the sub-picture data decoded by thesub-picture decoder 513 is temporarily recorded into the memory 540acting as a buffer. Then, at least one of the SP data (still picturedata) and the SP control data information (still picture control data)included in the temporarily recorded sub-picture data is read undercontrol by the control signal Sc5 from the system controller 520. Then,by acting the SP control information on the SP data, the still picturedisplay is performed as all or part of the video output.

As shown in FIG. 20(a), the SP control information 721 has a SCP headerand SF control parameters. The “SCP header” is made of a SP dataidentifier to specify the SP data corresponding to the control object ofthe SP control information 721, the information to indicate the recordposition of the SP data and other information. The “SF controlparameters” are made of a plurality of parameters to control the SP datafor each sub-frame unit (SF unit) which is a part of an image cut out asat least a part of an image defined by the SP data. More specifically,the “SF control parameters” include parameters to indicate the displaystart time point or the display end time point of the SF data by meansof the PTS (Presentation Time Stamp) and the like. Still morespecifically, the “SF control parameters” include various parameters toindicate various conditions, such as the display time, the cut out rangeof the sub-picture or the display position thereof, the scaling, therotation and so on.

As shown in FIG. 20(b), the SP data structure 722 has the structureinformation and the SP data (main body or entity of the still picturedata). The “structure information” is made of the identifier to identifyor specify the SP data, and other information such as the length of theSP data. The “SP data” has image data and the like in JPEG format or bitmap data format subjected to a run length encoding, for example.

Therefore, when the sub-picture data is reproduced, various reproductioncontrols are performed with using the sub-frame, on the basis of the SFcontrol parameter shown in FIG. 20(a), for each sub-frame unit obtainedby cutting out at least a part of the SP data shown in FIG. 20(b).

As shown in FIG. 21, the SP control information 721 and the SP datastructure 722 are packetized into a plurality of TS packets 146 (seeFIG. 10) and multiplexed. The TS packet 146 for storing therein the headportion of the SP control information 721 in the sub-picture structureis referred to as a “SCP”, and the TS packet 146 for storing therein thehead portion of the SP data structure 722 in the sub-picture structureis referred to as a “SPD”.

As shown in FIG. 21(a), both the SP control information 721 includingthe SCP and the SP data structure 722 may be regarded as one sub-picturestructure and divided into the plurality of TS packets 146. As shown inFIG. 21(b), the SP control information 721 including the SCP may beregarded as one sub-picture structure and divided into the plurality ofTS packets 146. As shown in FIG. 21(c), the SP data structure 722including the SPD may be regarded as one sub-picture structure anddivided into the plurality of TS packets 146.

In the embodiment, for example, with respect to the SP data in the SPdata structure 722 recorded on a SPD stream, the SP control informationrecorded on a SCP stream different from the SPD stream is operated, tothereby perform the reproduction control of the still picture. In thiscase, there may be only one or a plurality of SCP streams which operateswith respect to one SPD stream. Recording the two types of streams ontomutually different elementary streams allows efficient reproductioncontrol. Moreover, operating a plurality of SP control information on aplurality of SCP streams, with respect to the SP data on one SPD stream,allows more efficient reproduction control.

More specifically, as shown in FIG. 22, at a time point t11 duringreproduction of a video stream (Video 1) of “ES_PID=200”, the reading ofthe SP data (SPD1) on a SPD stream of “ES_PID=201” is started, and it isstored into the memory 540 of the information record/reproductionapparatus 500 (refer to FIG. 14). Then, the stored SP data is storeduntil a set end time point, for example, or is stored until the readinga next sub-picture is started.

In FIG. 22, on a SCP stream (SCP1) of “ES_PID=202”, SCP#1 a, SCP#1 b,SCP#1 c and SCP#1 d are provided in timing of a time point t21, a timepoint t22, a time point t23 and a time point t24, respectively. On a SCPstream (SCP2) of “ES_PID=203”, SCP#2 a, SCP#2 b and SCP#2 c are providedin timing of a time point t31, a time point t32 and a time point t33,respectively. On a SCP stream (SCP3) of “ES_PID=204”, SCP#3 a, SCP#3 b,SCP#3 c and SCP#3 d are provided in timing of a time point t41, a timepoint t42, a time point t43 and a time point t44, respectively.

However, in addition to such reproduction control of the still picture,it is possible that with respect to the SP data in the sub-picture datastructure recorded on the sub-picture stream, the SP control informationin the sub-picture structure recorded on the same stream is operated, tothereby perform the reproduction control of the still picture. Namely,both the SP control information 721 and the SP data structure 722 may berecorded into only one sub-picture stream, to thereby operate the SPcontrol information 721 to the SP data structure 722.

In any case, by sharing or using many times the SP data provided as thebit map data and JPEG data which have a large data amount, it ispossible to save a limited recording capacity on the disc, which allowsmore efficient reproduction and display processing. In addition, in anycase, it is possible to superimpose such a sub-picture onto a motionpicture or main-video which is based on the video data recorded inanother video stream.

Now, with reference to FIG. 23 to FIG. 27, an explanation is made on thedata structure of the SP control information. FIG. 23 to FIG. 27 consista series of tables to indicate a structure of the sub-picture controlpacket in which the SP control information of this embodiment is stored.Incidentally, in the series of tables shown from FIG. 23 to FIG. 27, thedata structure is shown in a plurality of hierarchical stages, focusingon the data structure of the embodiment of this invention. Therefore,items which does not directly relate to the embodiment of this inventionare indicated as “others” or “other information(s)”. However, new itemsmay be added if needed.

Firstly, as shown in FIG. 23, the sub-picture control packet (table) 810includes the SCP header 811 and the sub-frame control parameters 812, asan example of the SP control information. Incidentally, as mentionedabove, the sub-picture control packet 810 is made of a plurality of TSpackets in which SCP is disposed at a head portion of them on thecontrol information stream or the sub-picture stream.

In FIG. 23, as shown in an upper portion of the central column, the SCPheader (table) 811 includes the information about the sub-picturecontrol packet 810, specifically a sub-picture series identificationnumber, SPD (SP data) entry section PTS's and others.

In the SCP header 811, the sub-picture series identification numberdefines a continuity of the sub-pictures. Relating to this, a certainSPD and a group of SCP's acting on the SPD are referred to as asub-picture sequence. On the other hand, the SPD entry section PTSdefines each start PTS of each entry section where a head packet of thesub-picture data as the control object exists.

In FIG. 23, as shown in the lower part of the central column, thesub-frame control parameters (table) 812 are various parameters todefine a display mode of the sub-frame on the main-video, specificallythe sub-frame total number, the sub-frame total button number, thesub-frame informations 813 (sub-frame informations #1 to #n), the viewrectangle 814, the sub-frame highlight scheme and others.

In the sub-frame control parameters 812, the sub-frame total numberdefines a total number of the sub-frames cut out from the sub-pictureimage. The sub-frame total button number defines a total number of thebutton of the sub-frame formed by the sub-picture image. Each of thesub-frame informations 813 (the sub-frame informations #1 to #n)includes informations about sub-frames #1 to #n, respectively. The viewrectangle 814 includes information about a rectangular window (displaywindow) through which the sub-frame disposed in a virtual Cartesiancoordinate system is viewed. Furthermore, the sub-frame highlight schememay include a highlight value of a button image of the sub-frame and thecontrol information thereof, as an example of the highlight informationof the “SCP button” according to the present invention.

In FIG. 23, as shown in the upper part of the right column, each of thesub-frame informations 813 (sub-frame informations #1 to #n) includes asub-frame start PTS, a sub-frame end PTS, a sub-frame rectangle, aSFCCI, a sub-frame control components and so on.

In each of the sub-frame informations 813 (the sub-frame informations #1to #n), the sub-frame start PTS defines a PTS to start a display ofimage information designated by the sub-frame. The sub-frame end PTSdefines a PTS to end the display of the image information designated bythe sub-frame. The sub-frame rectangle defines a rectangular area to becut out for the sub-frame, for example by designating coordinates of theupper left corner and the lower right corner.

The SFCCI (Sub-Frame Control Component Information) is an example of the“type designation information” according to the present invention. Forexample, if a value “1” is assigned to each article, it means that thesub-frame can be displayed in the display mode indicated by thisarticle, or means that the control information corresponding to thisarticle exists. Alternatively, if a value “0” is assigned to eacharticle, it means that the sub-frame cannot be displayed in the displaymode indicated by this article, or means that the control informationcorresponding to this article does not exist.

Furthermore, the sub-frame control components indicate specifically atype for controlling the sub-frame display, as each article whoseexistence or effectiveness is indicated by the SFCCI. For example, thesub-frame control components indicate control type such as a 2D display,a button display, a moving display, an alpha-blending display, a shadingor shadow display and so on as a type for controlling the sub-framedisplay.

In FIG. 23, as shown in the lower part of the right column, the viewrectangle 814 has the information to indicate Vx, Vy, a width, a height,and a slope of the rectangle (view up vectors Ux and Uy). The viewrectangle is for controlling a position of the display window throughwhich one or more sub-frames disposed in the virtual Cartesiancoordinate system is viewed. Vx and Vy designate coordinates ofreference points (see FIG. 35) of the display window (for example, thelower left corner in this example). The width designates the width ofthe display window, and the height designates the height of the displaywindow. The view up vector designates the vector information to indicatea slope of the display window.

Now, as shown in FIG. 24, each of the sub-frame informations (table)813, i.e. each of the sub-frame informations #1 to #n, includes aplurality of tables.

In FIG. 24, as shown in the upper part of the central column, thesub-frame rectangle (table) 815 includes information to designate rangesof the sub-frame such as sub-frame x1, sub-frame y1, sub-frame x2 andsub-frame y2. This information indicates coordinates to cut outsub-frames from the sub-picture. The sub-frame x1 and the sub-frame y1correspond to the coordinate of an upper left corner of the range, andthe sub-frame x2 and the sub-frame y2 correspond to the coordinate of alower right corner of the range.

In FIG. 24, as shown in the upper part of the right column, a SFCCI bitallocation (table) 816 consists of 32 bits. Each bit indicates, by itsvalue “0” or “1”, the existence or effectiveness of the controlinformation relating to each display mode to display the sub-frame overthe main-video. For example, “bit 0” indicates the existence of thesub-picture control information to perform a 2D conversion or theeffectiveness of the 2D conversion. For example, “bit 1” indicates theexistence of the sub-picture control information to perform a buttondisplay (i.e. the SCP button display) or the effectiveness of the buttondisplay. For example, “bit 3” indicates the existence of the sub-picturecontrol information to perform an alpha-blending or the effectiveness ofthe alpha-blending. For example, “bit 4” indicates the existence of thesub-picture control information to perform a shadow dropping or theeffectiveness of the shadow dropping.

In FIG. 24, as shown in the lower part of the right column, thesub-frame control component (table) 817 has specific contents of varioussub-picture control informations whose existence or effectiveness isindicated by the SFCCI bit allocation table 816. For example, specificsub-picture control information including a 2D display, a button displayand so on is recorded.

Now, as shown in FIG. 25, the sub-frame control component (table) 817includes a plurality of tables.

In FIG. 25, as shown in the upper part of the central column, the 2Dconversion control component (table) 818 includes sub-frame controlcomponent identification information, TM11, TM12, TM21, TM22, TM31, TM32and so on. The sub-frame control component identification information isinformation to identify a component relating to the 2D conversion (e.g.an identification number assigned to the 2D conversion control). Each TMis information to designate a vertical element and a horizontal elementof the conversion matrix used for the 2D control. In accordance with the2D conversion control component 818 constructed as such, a 2D displaycan be achieved with using the sub-picture according to the embodiment.

In FIG. 25, as shown in the upper part of the right column, thesub-frame button control component (table) 819 includes sub-framecontrol component identification information, sub-frame initial buttonstatus, sub-frame button command and so on. The sub-frame controlcomponent identification information is information to identify acomponent relating to the sub-frame button (e.g. an identificationnumber assigned to the button control). The sub-frame initial buttonstatus indicates an operational condition of the button in its initialstatus. The sub-frame button command indicates a command to be executedin response to the button operation of the sub-frame. In accordance withthe sub-frame button control component 819 constructed as such, the SCPbutton display can be achieved with using the sub-picture according tothe embodiment.

In FIG. 25, as shown in the central part of the right column, the shadowdropping control component (table) 820 includes sub-frame controlcomponent identification information, shadow offset x, shadow offset y,shadow R, shadow G, shadow B and so on. Incidentally, it may includebrightness and color difference information, instead of the shadow R,the shadow G and the shadow B. The sub-frame control componentidentification information is information to identify a componentrelating to the shadow dropping (e.g. an identification number assignedto the shadow dropping control). The shadow offset x designates ahorizontal offset of a shadow based on the original image. The shadowoffset y designates a vertical offset of the shadow based on theoriginal image. Each offset may be defined by pixel unit for example.The shadow R, the shadow G and the shadow B designate color of theshadow. In accordance with the shadow dropping control component 820constructed as such, the shadow dropping display can be achieved aboutthe sub-picture according to the embodiment.

In FIG. 25, as shown in the lower part of the right column, thesub-frame rectangle motion control component (table) 821 includessub-frame control component identification information, pre-delay, postdelay, frame step, leg total number, spline control, leg informations #1to #n and so on. The sub-frame control component identificationinformation is information to identify a component relating to thesub-frame rectangle motion control (e.g. an identification numberassigned to the rectangle motion control). The leg total numberdesignates a number of passing points on the route or path along whichthe sub-frame traces. The spline control is information to tracesmoothly the passing points from the start point to the end point of thesub-frame's tracing. These passing points may be traced linearly, if asmooth trace as mentioned above is not indicated. The leg informations#1 to #n are informations about each passing point, respectively. Inaccordance with the sub-frame rectangle motion control component 821constructed as such, the rectangle motion can be achieved about thesub-picture according to the embodiment.

In FIG. 25, as shown in the lower part of the central column, the 2Dconversion motion control component (table) 822 includes sub-framecontrol component identification information, pre-delay, post delay,frame step, leg total number, spline control, leg informations #1 to #nand so on. The sub-frame control component identification information isinformation to identify a component relating to the 2D conversion motioncontrol (e.g. an identification number assigned to the 2D conversionmotion control). The pre-delay indicates a control to start the movementof the sub-frame with a predetermined delay from the display start timepoint of the sub-frame. The post delay indicates a control to start themovement with a predetermined delay from the display end time point ofthe sub-frame. The frame step designates a period or time cycle of theoperation. The leg total number is representative of the number ofpassing points on the path along which the sub-frame traces. The splinecontrol designates whether or not passing points from the start point tothe end point of the sub-frame's tracing is to be traced in a smoothedcurve. In the spline control, the path does not always trace rightly onpassing points, but may trace in the proximity of points. If the smoothtrace of such a spline form is not designated, a control is executed soas to trace these passing points linearly. The leg informations #1 to #nhave information about each passing point, respectively. In accordancewith the 2D conversion motion control component 822 constructed as such,the 2D conversion motion can be achieved about the sub-picture accordingto the embodiment.

Now, as shown in FIG. 26, the sub-frame rectangle motion controlcomponent (table) 821 includes a plurality of tables.

In FIG. 26, as shown in the right column, each of the leg informations(tables) 823, i.e. each of the leg informations #1 to #n, includesrelative location x, relative location y, relative scaling and so on.The relative location x indicates a horizontal position of the passingpoint relative to the current position. The relative location yindicates a vertical position of the passing point relative to thecurrent position. Furthermore, the relative scaling includes informationfor a scaling from the current size into a predetermined size. Inaccordance with the leg informations 823 constructed as such, thedisplay position or the display size of the sub-picture according to theembodiment can be changed.

Now, as shown in FIG. 27, the 2D conversion motion control component(table) 822 includes a plurality of tables.

In FIG. 27, as shown in the right column, each of the leg informations(table) 824. i.e. each of the leg informations #1 to #n, includes pivotpoint x, pivot point y, relative location x, relative location y,relative scaling, relative angle and so on. The pivot point is fordesignating a center for the rotation or scaling during the movement ofthe sub-frame. The pivot point x designates x coordinate of such acenter. The pivot point y designates y coordinate of such a center. Therelative location x indicates a horizontal position relationshiprelative to the current position. The relative location y indicates avertical position relationship relative to the current position.Furthermore, the relative scaling includes information for a scalingfrom the current size into a predetermined size. The relative angleincludes information for a conversion from the current angle into apredetermined angle. In accordance with the leg informations 824constructed as such, the display position or the display size of thesub-picture according to the embodiment can be changed.

As explained with reference to FIG. 23 to FIG. 27, the SP controlinformation stored in the sub-picture control packet according to theembodiment has a hierarchical data structure.

Now, an explanation is made on various display controls of the sub-framespecifically executed on the basis of the hierarchical data structureconstructed as mentioned above, with reference to FIG. 28 to FIG. 47.

Firstly, with reference to FIG. 28, an explanation is made on a controlto cut out the sub-frame from the sub-picture. FIG. 28 conceptuallyshows how to cut the sub-frame from the sub-picture.

In FIG. 28, the sub-picture 831 is recorded in advance as SP data (StillPicture data) and as a sub-picture stream. This sub-picture 831 isreproduced by the information record reproduction apparatus 500 and thenrecorded into the memory 540. The sub-frame 832 is an image part cut outfrom the sub-picture 831, on the basis of the SP control information(see FIG. 23 to FIG. 27) which is reproduced from an elementary streamthe same as or different from this sub-picture stream.

For example, in FIG. 28, the coordinates of the upper left corner aredefined as (x1, y1), and the coordinates of the lower right corner aredefined as (x2, y2). The sub-frame is defined as a rectangle having adiagonal line defined by these coordinates (x1, y1) and (x2, y2). Therange of the sub-frame is set by reading the sub-picture control packet810, by referring further to the sub-frame control parameter 812 in thesub-picture control packet 810, by referring further to the sub-frameinformations 813 (the sub-frame informations #1 to #n) in the sub-framecontrol parameter 812, by referring further to the sub-frame rectangle815 in the sub-frame information 813, and by referring further to thesub-frame x1, the sub-frame y1, the sub-frame x2 and the sub-frame y2 inthe sub-frame rectangle 815 (see FIG. 23 and FIG. 24).

Now, an explanation is made on a control to cut out a plurality ofdifferent sub-frames from the same sub-picture (SP data), with referenceto FIG. 29. FIG. 29 conceptually shows three specific examples to cutout one or more sub-frames from the same sub-picture.

In FIG. 29, as shown in the “example 1”, the sub-frame 832 may be cutout so as to be the same as the sub-picture 831.

As shown in the “example 2”, at least a part of the sub-picture 831 maybe cut out as a sub-frame 832 a (an upper figure of the “example 2”).Alternatively, a plurality of sub-frames 832 b, 832 c and 832 d may becut out separately from each other (a lower figure of the “example 2”).

As shown in the “example 3”, a plurality of sub-frames (e.g. a sub-frame832 a and a sub-frame 832 e) may be cut out so as to include some commonpart to each other, i.e. overlappingly (an upper figure of the “example3”). Alternatively, they may be cut out in such a manner that one (e.g.832 f) includes wholly the other (e.g. 832 g) (a lower figure of the“example 3”).

A variety of sub-frames as mentioned above is set by referring to thesub-frame x1, the sub-frame y1, the sub-frame x2 and the sub-frame y2included in the hierarchical structure of the sub-picture control packet810, as in the case of FIG. 28 (see FIG. 23 and FIG. 24). Thus, avariety of sub-frames can be cut out from the same sub-picture.

Now, an explanation is made on a recording format of the cut outsub-frame, with reference to FIG. 30. FIG. 30 conceptually shows adisplay position of the sub-frame.

As shown in FIG. 30, in the cut out processing of the sub-frame asmentioned above, it is performed that the cut out sub-frames 832 a to832 g are recorded after converting them into a sub-frame coordinatesystem (X_(L), Y_(L)). That is, in the information record reproductionapparatus 500, the coordinate systems of all the sub-frames 832, nomatter what range of the sub-picture is defined to cut out thesub-frames, are converted into the coordinate system shown in FIG. 30and recorded. In this example, the sub-frame 832 is recorded with thelower left corner thereof being as the origin of the coordinate system.

Now, an explanation is made on a conversion processing from thesub-frame coordinate system to the main-video virtual coordinate system,with reference to FIG. 31. FIG. 31 conceptually illustrates sub-framesin each sub-frame coordinate system and a disposition of a main-videocomposed of these sub-frames in the main-video virtual coordinatesystem.

As illustrated in FIG. 31, a plurality of sub-frames (left-half side ofFIG. 31) each of which is converted into the sub-frame coordinate system(X_(L), Y_(L)) and recorded is then converted into the main-videovirtual coordinate system(Xw, Yw) so that one image (right-half side ofFIG. 31) is synthesized. Herein, sub-frame images for tires and a bodyare converted into the main-video virtual coordinate system (Xw, Yw) anddisplayed so that an automobile image is synthesized. The SP controlinformation to control such a display is recorded in the sub-picturecontrol packet 810 (see FIG. 23 to FIG. 27). Depending on contents ofvarious control informations in the sub-picture control packet 210, itis possible to synthesize the image in such a manner that the rear tiremay be laid down aside of the body, or a manner that each tire rotates.

The display illustrated in shown in FIG. 31 is achieved with usingsub-frames for tires and the body predetermined in advance, for example,by reading the sub-picture control packet 810, then referring to thesub-frame control parameter 812 therein, then referring to the sub-frameinformation 813 therein and then referring to the sub-frame controlcomponents 817 therein (see FIG. 23). The disposition in the main-videoconfigured as such may be set arbitrary in the main-video virtualcoordinate system (Xw, Yw).

Now, an explanation is made on a 2D conversion motion control, withreference to FIG. 32. FIG. 32 conceptually shows a display schemeinvolving a shift (parallel displacement) of a sub-frame in a main-videocoordinate system.

As illustrated in FIG. 32, for the purpose of the 2D conversion motioncontrol, a freely selected sub-frame 837 is shifted in the main-videovirtual coordinate system (Xw, Yw) without modification the sub-frame837. The sub-frame 837 is cut out from a predetermined sub-picture.Displaying such a sub-frame 837 may be achieved by reading thesub-picture control packet 810, then referring to the sub-frame controlparameter 812 therein, then referring to the sub-frame information 813therein, then referring to the sub-frame control components 817 therein,and then referring to the 2D conversion motion component 822 (see FIG.23 to FIG. 27).

Now, an explanation is made on a control involving an image size changeof the sub-frame and a control involving a rectangle motion, withreference to FIG. 33. FIG. 33 conceptually shows a display schemeinvolving a shift and an image size change of a sub-frame in amain-video coordinate system.

As illustrated in FIG. 33, for the purpose of the image-size-changingcontrol and the 2D conversion motion control, a freely selectedsub-frame 837 is scaled down (or scaled up) and shifted to form an image838 in the main-video coordinate system (Xw, Yw). An initial sub-frame837 is cut out from a predetermined sub-picture. Such a displayinvolving a conversion from the image 837 to the image 838 may beachieved by reading the sub-picture control packet 810, then referringto the sub-frame control parameter 812 therein, then referring to thesub-frame information 813 therein, then referring to the 2D conversionmotion component 822 therein, then referring to the sub-frame rectanglemotion component 821 therein, and then referring to the leginformation(s) 824 therein (see FIG. 23 to FIG. 26). That is, on thebasis of the relative scaling and the like included in the leginformation(s) 824, the display involving the conversion from thesub-frame 837 to the image 838 is achieved. In this case, the leginformations 824 (i.e. leg informations #1 to #n) indicate only thestart point and the end point. In an area between the start point andend point, the display control is performed so that the sub-frame moveslinearly.

Now, an explanation is made on the 2D conversion motion controlinvolving a rotation of the sub-frame, with reference to FIG. 34. FIG.34 conceptually shows a display scheme involving a rotation of thesub-frame in the main-video virtual coordinate system.

As illustrated in FIG. 34, for the purpose of the 2D conversion motioncontrol involving a rotation, a freely selected sub-frame 837 is rotatedwithout modifying the sub-frame 837, in the main-video virtualcoordinate system (Xw, Yw). The initial sub-frame 837 is cut out from apredetermined sub-picture. Such a display involving a rotation of thesub-frames may be achieved by reading the sub-picture control packet810, then referring to the sub-frame control parameter 812 therein, thenreferring to the sub-frame information 813 therein, then referring tosub-frame control components 817 therein, then referring to the 2Dconversion motion control component 822 therein, then referring to theleg information (s)824 (leg informations #1 to #n), and then referringto the pivot point x, the pivot point y, the relative angle and the like(see FIG. 23 to FIG. 27).

Now, an explanation is made on a display window (view rectangle) controlof the sub-frame, with reference to FIG. 35. FIG. 35 conceptuallyillustrates a display scheme involving the display window and adisposition of the sub-frame in the main-video virtual coordinatesystem.

As illustrated in FIG. 35, a sub-frame(s) 837 (may be one or more) iscut out from a predetermined sub-picture and disposed at a predeterminedposition in the main-video virtual coordinate system (Xw, Yw).Processings until this disposition may be performed in accordance withthe information about the 2D conversion control component 818, or may beperformed by a scheme explained with reference to FIG. 32, FIG. 33, FIG.34 and the like. Furthermore, a view rectangle, i.e. a display window,is defined and moved in the main-video virtual coordinate system (Xw,Yw), resulting in a processing to display one or more images of thesub-frame(s) 837 disposed in the main-video virtual coordinate systemand included in an area encompassed within the view rectangle.Furthermore, such a display involving the view rectangle may be achievedby reading the sub-picture control packet 810, then referring to thesub-frame control parameter 812 therein, then referring to the viewrectangle 814 therein, and then referring to Vx, Vy, the width, theheight and the like therein (see FIG. 23).

Furthermore in this example, a control may be achieved so that the areaencompassed within the view rectangle is changed by changing Vx, Vy, thewidth, the height and the like over time.

Now, an explanation is made on a display control involving a rotationand an image size change of the sub-frame, with reference to FIG. 36.FIG. 36 conceptually shows a display scheme involving the rotation andthe image size change of the sub-frame in the main-video virtualcoordinate system.

As shown in FIG. 36, a freely selected sub-frame 837 is scaled downgradually or sequentially (or scaled up gradually or sequentially) androtationally moved in the main-video virtual coordinate system (Xw, Yw)to form an image 841 through images 839 and 840. The initial sub-frame837 is cut out from a predetermined sub-picture. Such a display of thesub-frames may be achieved by reading the sub-picture control packet810, then referring to the sub-frame control parameter 812 therein, thenreferring to sub-frame information 813 therein, then referring to thesub-frame control components 817 therein, then referring to the 2Dconversion motion control component 822 therein, then referring to theleg information(s) 824 (leg informations #1 to #n), and then referringto the pivot point x, the pivot point y, the relative location x, therelative location y, the relative scaling, the relative angle and thelike (see FIG. 23 to FIG. 27). Herein, images 839, 840 and 841 aredisposed on the basis of predetermined angles defined by the relativeangle, and displayed sequentially on the basis of the scaling defined bythe relative scaling.

Now, an explanation is made on a display control involving a movement ofthe sub-frame in the sub-picture, i.e. a sub-frame rectangle motiondisplay control, with reference to FIG. 37. FIG. 37 conceptuallyillustrates a display scheme involving a movement of the sub-frame inthe sub-picture.

As illustrated in FIG. 37, sub-frames 837, 842 and 843 are cut outsequentially from the sub-picture 831, in accordance with apredetermined rule. Then, these cut out sub-frames are displayedsequentially in the main-video virtual coordinate system (Xw, Yw). Sucha display of the sub-frames may be achieved by reading the sub-picturecontrol packet 810, then referring to the sub-frame control parameter812 therein, then referring to the sub-frame information 813 therein,then referring to the sub-frame control components 817 therein, thenreferring to the sub-frame rectangle motion control component 821therein, then referring to the leg information(s) 823 therein, and thenreferring to the relative location x, the relative location y, therelative scaling and the like relative to the initial location of thesub-frame designated by the sub-frame rectangle 815 (see FIG. 24 to FIG.26).

Now, an explanation is made on a display control involving a moving pathof the sub-frame in the main-video virtual coordinate system, withreference to FIG. 38. FIG. 38 conceptually shows a display schemeinvolving the moving path of the sub-frame in the main-video virtualcoordinate system.

As illustrated in FIG. 38, the sub-frame moves over the predeterminedpath. Particularly in FIG. 38(a), the sub-frame moves over the pathlinearly connecting from the start point to the end point through thepassing points 1 and 2. In FIG. 38(b), the sub-frame moves over the pathsubjected to a spline interpolation from the start point to the endpoint through passing points 1 and 2. The display information aboutthese passing points, the linear movement, the movement with the splineinterpolation and the like is recorded in the sub-picture control packet210. Furthermore, passing points are not limited to two in their number,but may be more. Such a display of the sub-frame may be achieved byreading the sub-picture control packet 812, then referring to thesub-frame control parameter 812 therein, then referring to the sub-frameinformation 813 therein, then referring to the sub-frame controlcomponents 817 therein, then referring to the sub-frame rectangle motioncontrol component 821 therein, or referring to the 2D conversion motioncomponent 822, and then referring to the spline control and the liketherein (see FIG. 23 to FIG. 25).

The SP data and the SP control information relating to various displayschemes discussed with reference to FIG. 23 to FIG. 38 is reproduced andread by the information record reproduction apparatus 500, as theelementary stream(s) recorded in the disc 100. The reading processingmay be performed as a part of the reproduction processing of the objectdiscussed with reference to the flow chart of FIG. 19. In this case, theSP data is cut out in accordance with the SP control information in thesub-picture control packet 210, and displayed over the main-video aftersubjected various manufacturings or processings as mentioned above.

In this embodiment, both display schemes are available, one is a buttondisplay based on the SP control information discussed with reference toFIG. 23 to FIG. 38 (i.e. the “SCP button” which is controlled on thebasis of the sub-frame button control component 819 of FIG. 25), theother is a button display based on the information provided in thestructure information acting as the header information of the SP datadiscussed with reference to FIG. 20 to FIG. 22 (i.e. the “SPD button”which is displayable and controllable based on the SP data structure 722of FIG. 20(b)). In this embodiment, either one of these “SCP button” and“SPD button” may be used, or both of them may be used as the situationdemands. In the following explanation, the button control information inthe header information (structure information) of the SP data requiredfor the “SPD button” is referred to as a “SPD button information” as thesituation demands, so as to be distinguished from the “SPD button”.

Now, an explanation is made on an example of obtaining drawing controlparameters, with reference to a flow chart of FIG. 39. FIG. 39 is a flowchart showing an operational flow of obtaining drawing controlparameters. Furthermore, in FIG. 39, the sub-picture control packet 810(see FIG. 23) is abbreviated as a “SCP”.

Incidentally, the “drawing control parameters” herein means parametersparticularly relating to drawing from among various sub-frame controlparameters, which compose the SP control information shown in FIG. 23,stored in the SP control packet. For example, all of the sub-framecontrol parameters 812 shown in FIG. 23 and the like may be the drawingcontrol parameters, or the former may includes other parametersdifferent from the latter.

Firstly, in FIG. 39, the number (N) of the SPD (SP data) buttons isobtained from the SCP header (step S101), and it is judged whether ornot the number (N) of the SPD buttons is larger than “0” (step S102). Ifit is judged that the number (N) of the SPD buttons is not larger than“0” at step S102 (step S102: No), it means that the SPD buttoninformation does not exist, and therefore the process goes to step S105.On the other hand, if it is judged that the number (N) of the SPDbuttons is larger than “0” at step S102 (step S102: Yes), all SPD buttoninformations (total number=N) are obtained (step S103), and thehighlight information of the SPD button is further obtained (step S104).This highlight information may be obtained from the header informationof the SP data in which the highlight information is recorded inadvance, or from the structure information of the SP data.Alternatively, the highlight information may be obtained from the SPcontrol information in which it is recorded in advance (e.g., in thiscase, the sub-frame highlight scheme is obtained from the sub-framecontrol parameter 812 shown in FIG. 23). Then, the number (M) of thesub-frames is obtained from the SP control information (step S105).

Then, the display start PTS and the display end PTS of the sub-frame areobtained from the sub-picture control parameters 812 (see FIG. 23) (stepS106), and the position information on the SP data of the sub-frame isobtained (step S107), and the SFCCI is checked and the sub-frame controlcomponent is obtained (step S108). Then, it is judged whether or not theinformations of all sub-frames (total number=M) are obtained (stepS109). If it is judged that the informations of all sub-frames are notobtained (step S109: No), the processings after step S106 are repeatedto obtain various informations of the sub-frame.

If it is judged that the information of all sub-frames are obtained(step S109: Yes), then it is judged whether or not the sub-frame buttonexists (step S110). If it is judged that the sub-frame button does notexist (step S110: No), the process goes to step S112. On the other hand,if it is judged that the sub-frame button exists (step S110: Yes), thehighlight information of the sub-frame is obtained (step S111). Then,the display object area is obtained (step S112), and obtaining thedrawing control parameters is terminated.

Thus, an explanation has been made on the operational flow of obtainingparameters relating to the SPD button and obtaining drawing controlparameters. Nevertheless, other parameters may be obtained in accordancewith procedures suitable for each of them.

Now, an explanation is made on an exemplarily operation of checking theSFCCI and obtaining the SFCC (sub-frame control component) at step S108of FIG. 39, with reference to a flow chart of FIG. 40. Incidentally, theSFCCI bit configuration herein may be bits 0 to 31 (LSB to MSB), asshown in the SFCCI bit allocation table 816 illustrated at the upperpart of the right column of FIG. 24.

Firstly, it is judged whether or not the SFCCI bit 0 is “1” (step S201).If it is judged that the bit 0 is not “1” (step S201: No), the processgoes to step S203. If it is judged that the bit 0 is “1” (step S201:Yes), it means that the conversion control component exists or iseffective, and therefore they are obtained (step S202).

Then, it is judged whether or not the SFCCI bit 1 is “1” (step S203). Ifit is judged that the bit 1 is not “1” (step S203: No), the process goesto step S205. If it is judged that the bit 1 is “1” (step S203: Yes), itmeans that the sub-frame button control component exists or iseffective, and therefore they are obtained (step S204).

Then, it is judged whether or not the SFCCI bit 4 is “1” (step S205). Ifit is judged that the bit 4 is not “1” (step S205: No), the process goesto step S207. If it is judged that the bit 4 is “1” (step S205: Yes), itmeans that the shadow dropping control component exists or is effective,and therefore they are obtained (step S206).

Then, it is judged whether or not the SCCI bit 8 is “1” (step S207). Ifit is judged that the bit 8 is not “1” (step S207: No), the process goesto step S209. If it is judged that the bit 8 is “1” (step S201: Yes), itmeans that the sub-frame rectangle motion control component exists or iseffective, and therefore they are obtained (step S208).

Then, it is judged whether or not the SFCCI bit 9 is “1” (step S209). Ifit is judged that the bit 9 is not “1” (step S209: No), a series ofprocessings ends. On the other hand, if it is judged that the bit 9 is“1” (step S209: Yes), it means that the conversion motion controlcomponent exists or is effective, and therefore they are obtained (stepS210) and then a series of processings ends.

As discussed above, the sub-frame control component whose existence oreffectiveness is determined by the SFCCI is obtained. In this example,the bit 0, bit 1, bit 4, bit 8 and bit 9 are checked, but other bitslisted in the SFCCI bit allocation table 816 (see FIG. 24) may be alsochecked in the same manner, if needed.

Now, an explanation is made on the sub-frame drawing processing, withreference to flow charts of FIG. 41 to FIG. 44. Incidentally, these flowcharts together form a series of flow charts, relationships among whichare indicated by numerals in circles such as {circle around (1)},{circle around (2)}, and so on.

Firstly in FIG. 41, it is judged whether or not the effective conversioncontrol component is obtained (step S301). If it is judged that theeffective conversion control component is obtained (step S301: yes), theparameter(s) obtained from the conversion control component is (are) setinto a coordinate conversion matrix of the sub-frame (step S302). If itis judged that the effective conversion control component is notobtained (step S301: No), a unit matrix is set as the coordinateconversion matrix of the sub-frame (step S303). Then, it is judgedwhether or not the effective sub-frame button control component isobtained (step S304).

If it is judged that the effective sub-frame button control component isobtained at step S304 (step S304: Yes), parameter(s) required for ahighlight corresponding to an initial status of the sub-frame button is(are) obtained (step S305). For example herein, the sub-frame highlightscheme (table) in the sub-frame control parameter 812 illustrated inFIG. 23 is obtained, and the identification number information and thelike of the button which is designated by this sub-frame highlightscheme and is activated at the initial status is obtained.

Then, a command to be executed when the sub-frame button is pressed isobtained (step S306) and the process goes to step S501 of FIG. 43.

On the other hand, if it is judged that the effective sub-frame buttoncontrol component is not obtained (step S304: No), the process goes tostep S401 of FIG. 42.

In FIG. 42, it is judged whether or not the effective SPD buttoninformation is obtained (step S401). If it is judged that the effectiveSPD button information is obtained (step S401: Yes), parameter(s)required for a highlight corresponding to the initial status of the SPDbutton is (are) obtained (step S402). Then, the command to be executedwhen the SPD button is pressed is obtained (step S403), the process goesto step S501 of FIG. 43. On the other hand, if it is judged that theeffective SPD button information is not obtained at step S401 (stepS401: No), the process goes to step S501 of FIG. 43.

Then, it is judged whether or not the effective shadow droppingcomponent is obtained at step S501 of FIG. 43 (step S501). If it isjudged that the effective shadow dropping component is obtained (stepS501: yes), the information of the object to be shadowed (step S502),and both or either of the horizontal offset information of the shadowand vertical offset information of the shadow is (are) obtained (stepS503), the color information of the shadow is obtained (step S504), andthe other information is obtained (step S505). Then, the shadow isdepicted in a background buffer (step S506), and the process goes tostep S601 of FIG. 44. On the other hand, if it is judged that theeffective shadow dropping component is not obtained at step S501 (stepS501: No), the process also goes to step S601 of FIG. 44.

Then, it is judged whether or not the effective conversion motioncontrol component is obtained at step S601 of FIG. 44 (step S601). If itis judged that the effective conversion motion control component isobtained (step S601: Yes), increment values of individual elements inthe conversion matrix are calculated, on the basis of the sub-framedisplay time information and parameter(s) of the conversion motioncontrol component (step S602), and the process goes to step S603. On theother hand, if it is judged that the effective conversion motion controlcomponent is not obtained at step S601 (step S601: No), the process alsogoes to step S603.

Then, it is judged whether or not the effective sub-frame rectanglemotion control component is obtained at step S603 (step S603). If it isjudged that the effective sub-frame rectangle motion control componentis obtained (step S603: Yes), increment values of the coordinates of thesub-frame to be moved, on the basis of the sub-frame display timeinformation and parameter(s) of the sub-frame rectangle motion controlcomponent (step S604), and the process goes to step S605. On the otherhand, if it is judged that the effective sub-frame rectangle motioncontrol component is not obtained at step S603 (step S603: No), theprocess also goes to step S605.

Then, an image of the sub-frame is depicted in the background buffer(step S605), and the depicted image in the background buffer is began tooutput to a foreground (step S607), when or after the presentreproduction time becomes to be equal to the display start PTS of thesub-frame (step S606). Then, it is judged whether or not the presentreproduction time becomes to be equal to the display end PTS of thesub-frame (step S608). If it is judged that the present reproductiontime becomes to be equal to the display end PTS, a series of depictingends.

On the other hand, in the case that the present reproduction timebecomes to be equal to the display end PTS at step S608 (step S608: No),if there is any parameter value obtained from the sub-frame rectanglemotion control component, the increment is added to the parameter (stepS609), and if there is any parameter increment value obtained from theconversion motion control component, the increment is added to theparameter (step S610), and if any change is given to the drawingparameter, an image of the sub-frame is depicted in the backgroundbuffer on the basis of the new parameter (step S611), and the processgoes back to step S607 to repeat the following processings.

Thus, as discussed with reference to FIG. 41 to FIG. 44, a series ofdepicting processings of the sub-frame is performed. Incidentally, thedepicting processing is not limited to the above illustration, but cantake any method including any processing article, any processingsequence and so on, insofar as the same processing as the aboveillustration can be achieved.

(Exemplary Display of Shadow Dropping)

Now, an explanation is made on a display control with “shadow dropping”to shadow letters or characters to be displayed or shadow the sub-frame,with reference to FIG. 45. FIG. 45 conceptually illustrates the shadowdropping display.

In FIG. 45, two exemplary displays at the left side indicate shadowdroppings relative to sub-frames having opaque letters in transparentbackgrounds, respectively, while two exemplary displays at the rightside indicate shadow droppings relative to sub-frames entirely opaque.Furthermore, two exemplary displays at the upper side indicate shadowdroppings without adding blurring relative to the shadow, while twoexemplary displays at the lower side indicate shadow dropping withadding blurring relative to the shadow.

In the case that the shadow is given to opaque letters in thetransparent background of the sub-frame, these letters are displayed asletters or designs isolated stereoscopically over the main-video. In thecase that the sub-frame is opaque as a whole, an image in which theshadow is given to the entire sub-frame or the outline of the sub-frameis displayed over the main-video. Incidentally, “alpha value” may beused to indicate the transparency, which may be binary or multilevel(e.g. 255 levels). In the case of binary value, value “0” may be definedas “transparent”, and value “1” may be defined as “opaque”. In the caseof multilevel value, value “0” may be defined as “transparent”, the maxvalue which may be “255” may be defined as “opaque”, and mid values maybe defined to indicate appropriate transparency between transparent andopaque.

Furthermore, for the purpose of shadowing, the effective shadowing maybe achieved by applying an offset value on the opaque part and therebyadding a direction and volume or amount to be shadowed, or bycontrolling an angle of the background to be projected. Furthermore, theshadow is not limited to black color, but may take any desired color bymixing RGB signals in any desired proportion. Furthermore, the effectiveshadow may be obtained from the transparency of the entire sub-frame towhich an appropriate shadow is added.

The display control with shadow dropping as mentioned above is describedin the shadow dropping control component 820 in the sub-frame controlcomponents 817 shown in FIG. 25. That is, the information required for aspecific shadow dropping, including the offset amount or offsetposition, the shadow color and so on, can be obtained from thiscomponent, and the shadow is added on the basis of these informations.Then, by referring to the shadow dropping of the bit 4 in the SFCCI bitallocation table 816 shown in FIG. 24, the existence or effectiveness ofthe shadow dropping control component 820 is determined.

Furthermore, there is description about obtaining the controlinformation about the shadow dropping at step S206 in the operationalflow of obtaining the sub-frame control component and checking the SFCCIshown in FIG. 40, and there is description about a process to shadow asshown in FIG. 45 at steps S501 to S506 of FIG. 43.

(Exemplary Display of Button)

An explanation is made on a display control for “buttons”, withreference to FIG. 46. FIG. 46(a) shows a sub-frame and a range of abutton image. FIG. 46(b) shows an image based on an alpha value of thebutton. FIG. 46(c) shows an area acting as the button.

For example, as shown in FIG. 46(a), a part of the SP data including abutton design may be cut off as a sub-frame. The button information(i.e. the button position information to define the button range) tomake this sub-frame act as a button may be included in the headerinformation added to the SP data as for the “SPD button”, and may beincluded in the SP control information as for the “SCP button”.

On the basis of such button information, a function of the button asshown in FIG. 46 is determined. For example, types of the buttonfunction include a push button, a toggle button, an exclusive button andso on. Operation of these buttons are performed by a remote controloperation, an audio input operation, a touch operation on a screen, akeyboard operation and so on relative to the displayed button image.

In this case, the entire sub-frame does not always act as a button, butan only opaque area of the sub-frame defined by the alpha value may becut out as shown in FIG. 46(c) to make it act as the button image part(i.e. operative area as the button). That is, in this case, the alphavalue is referred to, and only a coordinate area indicated as opaque bythis alpha value is controlled to act as the button in accordance withthe aforementioned button information. For the simplicity, however, itis possible to make the entire sub-frame shown in FIG. 46(a) act as thebutton.

A highlight control may be applied to such a button. For example,highlight control is applied to such a button, in accordance with thesub-frame highlight scheme in the sub-frame control parameter 812 shownin FIG. 23, or, instead of or in addition to this, the sub-frame buttoncommand in the sub-frame button control component 819 shown in FIG. 25,or other information. The highlight display facilitates a distinctiverecognition about a presently pressed button, a presently selectedbutton, a presently selectable but non-selected button, a presentlynon-selectable button and so on. It is possible to display with apredetermined highlight not only the pressed button, but also aselectable button group in accordance with the present operatingcondition of the device. The highlight display applied to variousstatuses of the button can make the button statuses known to the user.

The highlight display control defined by the sub-frame highlight schemeand the like is performed in such a manner that the button to behighlightened becomes distinctive from other buttons or other parts orportions, for example by changing the brightness or contrast dependingon the button status, or giving a reverse shadow.

As for the SCP button, there is a description about the button displaycontrol mentioned above in the button information including thesub-frame button total number in the sub-frame control parameter 812shown in FIG. 23, the sub-frame initial button status in the sub-framebutton control component 819 shown in FIG. 25, the sub-frame buttoncommand and the like. Then, the existence or effectiveness to thesub-frame component 819 is determined by referring to the sub-framebutton information of the bit 1 in the SFCCI bit allocation table 816shown in FIG. 24.

On the other hand, as for the SPD button, there is a description aboutthe display control of the button mentioned above in the buttoninformation added to the SP data shown in FIG. 20(b). More specifically,for example, there is the description about that in the structureinformation in the SP data structure 722 shown in FIG. 20(b), or in“other information” in the SP data.

Furthermore, also the highlight information may be recorded (described)in the SP control information in association with each buttoninformation, as for both the SCP button and the SPD button.Alternatively, it may be recorded in the header information (or thestructure information or other information) in each SP data, inassociation with each button information. In any case, by performing thecontrol based on the button information and the highlight control basedon the highlight information in association with each other, the buttoncan be highlightened.

Furthermore, there is a description about a process to perform thebutton control illustrated in FIG. 46 in steps S101 to S104, steps S110to S111 in the operational flow of obtaining drawing control parameterof FIG. 39 and in steps S203 to S204 in the operational flow ofobtaining the sub-frame control component and checking the SFCCI of FIG.40, and further in steps S304 to S306 in the operational flow of thesub-frame drawing process of FIG. 41, and in steps S401 to S403 of FIG.42.

Now, an explanation is made on a transition of the buttons status, withreference to FIG. 47. Operational commands are predeterminedcorresponding to each of buttons, so that the information recordreproduction apparatus 500 performs by being operated a button matchedwith a desired operation. For example, a button relating to the audiofunction is selected from among displayed buttons and inputted, therebyto immediately perform the operation relating to the audio function.Types of operational buttons include a push button, a toggle button, anexclusive button and so on, as mentioned above.

The transition of the operational status of each button is determineddepending on the present operational status of each button. Thetransition of button status may include “inoperative”, “operative andpresently non-selected”, “operative and presently selected” and“executing”. The status “inoperative” means a status that the button isdisplayed but inoperative, the status “operative and presentlynon-selected” means a status that the button is operative but presentlynot selected, and thereby impossible to be directly operated, the status“operative and presently selected” means a status that the button isoperative and only this button is selected, and thereby possible to bedirectly operated, and the status “executing” means a status that theoperation of the button is presently executed. Due to the user'soperation, the status “operative and presently selected” is transited tothe status “executing” and the status “operative and presentlynon-selected”. Due to the system command, the status “operative andpresently selected” is transited to the status “inoperative”. Due to thesystem command, the status “inoperative” is transited to the status“operative and presently non-selected” and the status “operative andpresently selected”. Due to the user's operation or automatically afterthe command operation ends, the status “executing” is transited to thestatus “operative and presently selected”.

The button control shown in FIG. 47 may be performed in accordance withthe button information, as mentioned above, including the sub-framebutton total number in the sub-frame control parameter 812 shown in FIG.23, the sub-frame initial button status in the sub-frame button controlcomponent 819 shown in FIG. 25, the sub-frame button command and so on.

As discussed with reference FIG. 20 to FIG. 47, in this embodiment,various display controls are achieved, including the sub-picture motioncontrol, the shadow dropping control, the highlight button control andso on. Furthermore, these various display controls of sub-picture can beefficiently performed by using the control information such as theSFCCI.

(Access Flow on Reproduction)

Next, with reference to FIG. 48, the access flow on reproduction by theinformation record reproduction apparatus 500, employing the AUinformation 132 and the PU information 302, is explained as one of thefeatures of this embodiment, with the logical structure of the opticaldisc 100. FIG. 48 schematically illustrates an entire access flow onreproduction, in relation to the logical structure of the optical disc100.

In FIG. 48, the logical structure of the optical disc 100 is categorizedroughly into the following three hierarchies: a logic hierarchy 401; anobject hierarchy 403; and a logic-object association hierarchy 402mutually associating those two hierarchies.

Among them, the logic hierarchy 401 is a hierarchy for logicallyspecifying various logical information to reproduce the desired titleduring the reproduction, as well as the playlist (P list) to bereproduced and its constitutional contents. In the logic hierarchy 401,disc information 110 d indicating the entire titles 200 and the like onthe optical disc 100 is written within the disc information file 110(see FIG. 3), and further, reproduction sequence information 120 d ofthe entire contents on the optical disc 100 is written within theplaylist information file 120 (see FIG. 3). More specifically, theconstruction of one or more playlist sets 126S is written, as thereproduction sequence information 120 d, respectively for one or moretitle elements 200-2 included in each title 200. Furthermore, eachplaylist set 126S includes one or more playlists 126, in each of whichthe construction of one or more items 204 (see FIG. 13) is written.Then, at the time of the access during the reproduction, the logichierarchy 401 as described above specifies the title 200 to bereproduced, the playlist 126 corresponding to this, and further the item204 corresponding to this.

Next, the logic-object association hierarchy 402 is a hierarchy forspecifying the attribute and the physical storage address of the TSobject data 140 d to be reproduced, so as to specify the combinationand/or the construction of the TS object data 140 d as the entity dataand perform an address conversion to the object hierarchy 403 from thelogic hierarchy 401, on the basis of the information specified in thelogic hierarchy 401 as described above. More specifically, in thelogic-object association hierarchy 402, the object information data 130d, which separates a group of the contents composing each item 204 intounits of the AU 132 and which finely separates each AU 132 into units ofthe PU 302, is written in the object information file 130 (see FIG. 3).

Here, the “PU (Presentation Unit) 302” is a unit associating and unitinga plurality of elementary streams by the reproduction switchable unit.If there are three audio streams in the PU 302, the user can freelyswitch these three audio (e.g. audio for each language), during thereproduction of this vision.

On the other hand, the “AU (Associate Unit) 132” is a unit uniting aplurality of elementary streams such as video streams in the TS objectused in one title, and made of one or more PU's 302. More specifically,it is a unit uniting the elementary stream packet IDs (ES_PID) for eachTS object, indirectly via the PU 302. This AU 132 corresponds to anassembly made of a plurality of programs having a specificinter-relationship in view of the contents, such as a plurality ofprograms switchable to each other in the multi-source broadcasting. ThePU's 302 belonged to the same AU 132 corresponds to one or moreelementary stream assemblies each constructing a plurality of programsswitchable to each other by the user operation during the reproduction.

Therefore, if the AU 132 to be reproduced is identified, and the PU'sbelonged to the AU are identified, the elementary stream to bereproduced is identified. That is, a desired elementary stream can bereproduced from the multi-recorded optical disc 100, without using thePAT or the PMT shown in FIG. 12.

Incidentally, a further specific data structure of the AU information132I and the PU information 302I, each defining the AU 132 and the PU302, is discussed later in detail.

The elementary stream to be actually reproduced herein is identified ordesignated by the ES_PID that is a packet ID (see FIG. 12) of theelementary stream, on the basis of the PU information 302. At the sametime, the information indicating the start time point and the end timepoint of the reproduction is converted to the elementary stream addressinformation, and thereby the contents in a specific area (or a specifictime range) of a specific elementary stream is reproduced.

Thus, in the logic-object association hierarchy 402, an addressconversion is performed from a logical address relating to each item 204to a physical address relating to each PU 302.

Next, the object hierarchy 403 is a physical hierarchy to reproduce theactual TS object data 140d. In the object hierarchy 403, the TS objectdata 140 d is written within the object data file 140 (see FIG. 3). Morespecifically, TS packets 146 constructing a plurality of elementarystreams (ES) are multiplexed at every time point. The multiplexedpackets are disposed on the time scale to form a plurality of elementarystreams (see FIG. 11). Then, a plurality of TS packets multiplexed ateach time point is associated with a PU 302 identified by thelogic-object association hierarchy 402, for each elementary stream.Incidentally, it is possible to associate a plurality of PU's 302 withone elementary stream (e.g. one elementary stream relating to the sameaudio data is shared, or one elementary stream relating to the samesub-picture data is shared, among a plurality of switchable programs).

Thus, in the object hierarchy 403, the object data is actuallyreproduced, using the physical address obtained from the conversion inthe logic-object association hierarchy 402.

As described above, the three hierarchies shown in FIG. 48 allow makingan access to the optical disc 100 during the reproduction.

(Structure of Object Information File)

Next, with reference to FIG. 49, an explanation is made on a specificexample of the data structure in the object information file 130 toassociate the object data in the object data file 140 with variouslogical informations in the playlist information file 120 and the discinformation file 110 as discussed with regard to FIG. 48. FIG. 49schematically shows one specific example of the data structures of theAU (Associate Unit) table 131 (refer to FIG. 3) constructed in theobject information file 130 and the ES (Elementary Stream) map table 134(refer to FIG. 3) related to the AU table 134.

In this specific example, as shown in FIG. 49, the object informationtable is stored in the object information file 130. The objectinformation table is provided with the AU table 131 shown in the upperpart of FIG. 49 and the ES map table 134 shown in the lower part of FIG.49.

In the upper part of FIG. 49, the AU table 131 may have a structure thatallows the required number of tables for each Field to be added. Forexample, if there are four AUs, it may have such a structure that thenumber of the Fields increases to four.

In the AU table 131, there are stored “AU table general information” inwhich the number of AUs and the pointer to each AU, and the like arewritten, and “the other information.”

The AU table 131 describes therein the Index number (Index number=. . .) of the corresponding ES map table 134, as the AU information 132Iwhich indicates an ES table Index #m in each PU #m corresponding to eachAU #n. Here, the “AU” is a unit corresponding to a “program” in TVbroadcast, for example, as mentioned above (especially, in the case of“multi-vision” broadcasting, it is a unit of a group of a plurality of“visions” which is changeable or selectable), and it includes one ormore PUs, each of which is a reproduction unit. Moreover, the “PU” is agroup of mutually changeable elementary streams which are included ineach AU, as described above, and the ES table Index #m corresponding toeach PU is specified by the PU information 302I. For example, ifmulti-view contents are provided with the AU, the AU stores therein aplurality of PUs, and each PU stores therein the pointers to a pluralityof elementary stream packet IDs which indicates the packets constitutingthe contents of each view. This indicates the Index number in the ES maptable 134, as described later.

In the lower part of FIG. 49, in the ES map table 134, there are storedES map table general information, a plurality of Indexes #m (m=1, 2, . .. ), and the “other information”, for each Field.

The “ES map table general information” describes therein the size of theES map table, the total number of Indexes, and the like.

The “Index #m” includes the elementary stream packet ID (ES_PID) of theentire elementary stream to be used for the reproduction, thecorresponding Index number and the address information of the elementarystream.

In the embodiment, for example, if the elementary stream is the videostream of the MPEG 2 as described above, only the TS packet number ofthe TS packet at the head of the I picture and the corresponding displaytime length are written, as the address information, i.e. the ES addressinformation 134 d, on the ES map table 134, by which the data amount istried to be reduced. On the other hand, with regard to the ES addressinformation 134 d of the sub-picture stream, the data amount is tried tobe reduced by recording the index number information 134e of theelementary stream of the SP data to which the SP control information isapplied.

Because of the construction as described above, it is possible to obtainthe elementary stream packet ID (ES_PID) of the actual elementarystream, from the Index number of the ES map 134 specified from the AUtable 131. Moreover, since the address information of the elementarystream corresponding to the elementary stream packet ID can be obtainedat the same time, it is possible to reproduce the object data on thebasis of these information.

According to the data structure of the optical disc 100 explained above,even in adding a new title to the optical disc 100, necessaryinformation can be easily added, which is useful. On the other hand,even if some information becomes unnecessary as a result of editing orthe like, for example, what is to be done is simply not to refer to theinformation, and it is not necessary to actually delete the informationfrom the table, which is useful, as well.

As explained in detail with reference to FIG. 1 to FIG. 49, according tothis embodiment, the complex and sophisticated reproduction can beachieved to reproduce the sub-picture in association with the videodata, by using the SP control information and so on, the complex andsophisticated reproduction including an interactive reproduction withthe information record reproduction apparatus 500, a reproduction withshadowing the sub-picture, a highlight display control depending on acomplex and sophisticated button operation on a video screen and so on.Furthermore, it is possible to perform relatively efficiently andquickly these various display controls of the sub-picture by using thecontrol information such as the SFCCI from among the SP controlinformation.

Incidentally, in the aforementioned embodiment, the explanation is madeon the optical disc 100 as an example of the information record mediumand the recorder or player of the optical disc 100 as an example of theinformation record reproduction apparatus. Nevertheless, the presentinvention is not limited to the optical disc and the player or recorderthereof, but is applicable to various record media and the recorders orplayers thereof, supporting other high density recording or hightransfer rate.

The present invention is not limited to the above-described embodiments,and various changes may be made, if desired, without departing from theessence or spirit of the invention which can be read from the claims andthe entire specification. An information record medium, an apparatus forand a method of recording the information, an apparatus for and a methodof reproducing the information, an apparatus for and a method ofrecording and reproducing the information, a computer program forcontrolling the record or the reproduction, and a data structureincluding a control signal, all of which involves such changes, are alsointended to be within the technical scope of the present invention.

INDUSTRIAL APPLICABILITY

An information record medium, a apparatus for and a method of recordingthe information, an apparatus for and a method of reproducing theinformation, an apparatus for and a method of recording and reproducingthe information, a computer program for controlling the record or thereproduction, and a data structure including a control signal, all ofwhich are according to the present invention, can be applied to ahigh-density optical disc for consumer or industrial use, such as a DVD,on which various information, such as the main-video information, theaudio information and the sub-video information, can be recorded at highdensity and further can be applied to a DVD player, a DVD recorder, andthe like. Moreover, they can be applied to an information record medium,an information record reproduction apparatus, or the like, which aremounted on or can be connected to various computer equipment forconsumer or industrial use, for example.

1-17. (canceled)
 18. An information record medium on which there arerecorded: video information to indicate a main-video; sub-videoinformation to indicate a sub-video displayable at least partially overthe main-video; and sub-video control information including (i) acontrol information element for display control of the sub-videoinformation in a method set in advance and (ii) information to indicatepresence or absence, or effectiveness or ineffectiveness of the controlinformation element.
 19. The information record medium according toclaim 18, wherein the sub-video control information includes (i) aplurality of types of control information elements for display controlof the sub-video information in various methods set in advance and (ii)type indicating information to indicate presence or absence, oreffectiveness or ineffectiveness of the control information elements byeach of the types.
 20. The information record medium according to claim19, wherein the sub-video control information further includes sub-framerange information to specify at least a partial area of the sub-video asa sub-frame, the control information element comprises information forcontrolling the sub-video information by a unit of the sub-frame, andthe type indication information indicates the presence or absence, orthe effectiveness or ineffectiveness of the control information elementsby the unit of the sub-frame.
 21. The information record mediumaccording to claim 19, wherein the type indication information comprisestable information including one bit of information by each of the types.22. The information record medium according to claim 21, wherein thetable information includes extension bit information which is notassociated with the types, and a part occupied by the controlinformation elements in the sub-video control information isvariable-length.
 23. The information record medium according to claim18, wherein the control information element includes dynamic controlinformation to display the sub-video information dynamically.
 24. Theinformation record medium according to claim 18, wherein the controlinformation element includes shading-display control information toselectively shade an opaque part out of the sub-video and display itover the main-video.
 25. The information record medium according toclaim 18, wherein the control information element includes (i) buttoninformation which defines at least a part of the sub-video which is tobe displayed over the main-video, as a button video part which functionsas a button, and (ii) highlight information which defines how to performhighlight-display control over the button video part.
 26. Theinformation record medium according to claim 18, wherein the main-videoinformation, the sub-video information and the sub-video controlinformation are divided into predetermined packet units and multiplexed,and further streamed relatively into a video stream composed of thedivided main-video information, a sub-picture stream composed of thedivided sub-video information set and a control information streamcomposed of the divided sub-video control information.
 27. Aninformation record apparatus comprising: a first record device forrecording video information to indicate a main-video; a second recorddevice for recording sub-video information to indicate a sub-videodisplayable at least partially over the main-video; and a third recorddevice for recording sub-video control information including (i) acontrol information element for display control of the sub-videoinformation in a method set in advance and (ii) information to indicatepresence or absence, or effectiveness or ineffectiveness of the controlinformation elements.
 28. The information record apparatus according toclaim 27, wherein the third record device records the sub-video controlinformation includes (i) a plurality of types of control informationelements for display control of the sub-video information in variousmethods set in advance and (ii) type indicating information to indicatepresence or absence, or effectiveness or ineffectiveness of the controlinformation elements by each of the types.
 29. An information recordmethod comprising: a first record process of recording video informationto indicate a main-video; a second record process of recording sub-videoinformation to indicate a sub-video displayable at least partially overthe main-video; and a third record process of recording sub-videocontrol information including (i) a control information element fordisplay control of the sub-video information in a method set in advanceand (ii) information to indicate presence or absence, or effectivenessor ineffectiveness of the control information elements.
 30. Theinformation record method according to claim 29, wherein the thirdrecord process records the sub-video control information includes (i) aplurality of types of control information elements for display controlof the sub-video information in various methods set in advance and (ii)type indicating information to indicate presence or absence, oreffectiveness or ineffectiveness of the control information elements byeach of the types.
 31. An information reproduction apparatus forreproducing information on an information record medium on which thereare recorded: video information to indicate a main-video; sub-videoinformation to indicate a sub-video displayable at least partially overthe main-video; and sub-video control information including (i) acontrol information element for display control of the sub-videoinformation in a method set in advance and (ii) information to indicatepresence or absence, or effectiveness or ineffectiveness of the controlinformation element, said information reproduction apparatus comprising:a reproduction device for reproducing the video information, thesub-video information and the sub-video control information; a displayoutput device capable of displaying and outputting the reproducedsub-video information over the reproduced video information; and acontrol device for controlling the display output device todisplay-control the sub-video information in the method and display thesub-video information over the main-video, on the basis of the controlinformation element whose presence or effectiveness is shown by theinformation to indicate presence or absence, or effectiveness orineffectiveness of the control information element which is included inthe reproduced sub-video control information.
 32. The informationreproduction apparatus according to claim 31, wherein the sub-videocontrol information includes (i) a plurality of types of controlinformation elements for display control of the sub-video information invarious methods set in advance and (ii) type indicating information toindicate presence or absence, or effectiveness or ineffectiveness of thecontrol information elements by each of the types, the control devicecontrols the display output device to display-control the sub-videoinformation in the various methods and display the sub-video informationover the main-video, on the basis of the control information elementwhose presence or effectiveness is shown by the type indicationinformation which is included in the reproduced sub-video controlinformation.
 33. An information reproduction method of reproducinginformation on an information record medium on which there are recorded:video information to indicate a main-video; sub-video information toindicate a sub-video displayable at least partially over the main-video;and sub-video control information including (i) a control informationelement for display control of the sub-video information in a method setin advance and (ii) information to indicate presence or absence, oreffectiveness or ineffectiveness of the control information elements,said information reproduction method comprising: a reproduction processof reproducing the video information, the sub-video information and thesub-video control information; and a control process of controlling adisplay output device capable of displaying and outputting thereproduced sub-video information over the reproduced video information,to display-control the sub-video information in the method and displaythe sub-video information over the main-video, on the basis of thecontrol information element whose presence or effectiveness is shown bythe information to indicate presence or absence, or effectiveness orineffectiveness of the control information element which is included inthe reproduced sub-video control information.
 34. The informationreproduction method according to claim 33, wherein the sub-video controlinformation includes (i) a plurality of types of control informationelements for display control of the sub-video information in variousmethods set in advance and (ii) type indicating information to indicatepresence or absence, or effectiveness or ineffectiveness of the controlinformation elements by each of the types, the control process controlsthe display output device to display-control the sub-video informationin the various methods and display the sub-video information over themain-video, on the basis of the control information element whosepresence or effectiveness is shown by the type indication informationwhich is included in the reproduced sub-video control information. 35.An information record reproduction apparatus comprising: a first recorddevice for recording video information to indicate a main-video; asecond record device for recording sub-video information to indicate asub-video displayable at least partially over the main-video; a thirdrecord device for recording sub-video control information including (i)a control information element for display control of the sub-videoinformation in a method set in advance and (ii) information to indicatepresence or absence, or effectiveness or ineffectiveness of the controlinformation elements; a reproduction device for reproducing the videoinformation, the sub-video information and the sub-video controlinformation; a display output device capable of displaying andoutputting the reproduced sub-video information over the reproducedvideo information; and a control device for controlling the displayoutput device, to display-control the sub-video information in themethod and display the sub-video information over the main-video, on thebasis of the control information element whose presence or effectivenessis shown by the information to indicate presence or absence, oreffectiveness or ineffectiveness of the control information elementwhich is included in the reproduced sub-video control information. 36.The information record reproduction apparatus according to claim 35,wherein the third record device records the sub-video controlinformation includes (i) a plurality of types of control informationelements for display control of the sub-video information in variousmethods set to advance and (ii) type indicating information to indicatepresence or absence, or effectiveness or ineffectiveness of the controlinformation elements by each of the types, the control device controlsthe display output device to display-control the sub-video informationin the various methods and display the sub-video information over themain-video, on the basis of the control information element whosepresence or effectiveness is shown by the type indication informationwhich is included in the reproduced sub-video control information. 37.An information record reproduction method comprising: a first recordprocess of recording video information to indicate a main-video; asecond record process of recording sub-video information to indicate asub-video displayable at least partially over the main-video; a thirdrecord process of recording sub-video control information including (i)a control information element for display control of the sub-videoinformation in a method set in advance and (ii) information to indicatepresence or absence, or effectiveness or ineffectiveness of the controlinformation elements; a reproduction process of reproducing the videoinformation, the sub-video information and the sub-video controlinformation; and a control process of controlling a display outputdevice capable of displaying and outputting the reproduced sub-videoinformation over the reproduced video information, to display-controlthe sub-video information in the method and display the sub-videoinformation over the main-video, on the basis of the control informationelement whose presence or effectiveness is shown by the information toindicate presence or absence, or effectiveness or ineffectiveness of thecontrol information element which is included in the reproducedsub-video control information.
 38. The information record reproductionmethod according to claim 37, wherein the third record process recordsthe sub-video control information includes (i) a plurality of types ofcontrol information elements for display control of the sub-videoinformation in various methods set in advance and (ii) type indicatinginformation to indicate presence or absence, or effectiveness orineffectiveness of the control information elements by each of thetypes, the control process controls the display output device todisplay-control the sub-video information in the various methods anddisplay the sub-video information over the main-video, on the basis ofthe control information element whose presence or effectiveness is shownby the type indication information which is included in the reproducedsub-video control information.
 39. A computer program product for arecord control, to control a computer disposed at an information recordapparatus comprising: a first record device for recording videoinformation to indicate a main-video; a second record device forrecording sub-video information to indicate a sub-video displayable atleast partially over the main-video; and a third record device forrecording sub-video control information including (i) a controlinformation element for display control of the sub-video information ina method set in advance and (ii) information to indicate presence orabsence, or effectiveness or ineffectiveness of the control informationelement, said computer program product making the computer function asat least a part of the first record device, the second record device andthe third record device.
 40. A computer program product for areproduction control to control a computer disposed at an informationreproduction apparatus for reproducing information on an informationrecord medium on which there are recorded: video information to indicatea main-video; sub-video information to indicate a sub-video displayableat least partially over the main-video; and sub-video controlinformation including (i) a control information element for displaycontrol of the sub-video information in a method set in advance and (ii)information to indicate presence or absence, or effectiveness orineffectiveness of the control information elements, said informationreproduction apparatus comprising: a reproduction device for reproducingthe video information, the sub-video information and the sub-videocontrol information; a display output device capable of displaying andoutputting the reproduced sub-video information over the reproducedvideo information; and a control device for controlling the displayoutput device to display-control the sub-video information in the methodand display the sub-video information over the main-video, on the basisof the control information element whose presence or effectiveness isshown by the information to indicate presence or absence, oreffectiveness or ineffectiveness of the control information elementwhich is included in the reproduced sub-video control information, saidcomputer program product making the computer function as at least a partof the reproduction device, the display output device and the controldevice.
 41. A computer program product for a record reproduction controlto control a computer disposed at an information record reproductionapparatus comprising: a first record device for recording videoinformation to indicate a main-video; a second record device forrecording sub-video information to indicate a sub-video displayable atleast partially over the main-video; a third record device for recordingsub-video control information including (i) a control informationelement for display control of the sub-video information in a method setin advance and (ii) information to indicate presence or absence, oreffectiveness or ineffectiveness of the control information elements; areproduction device for reproducing the video information, the sub-videoinformation and the sub-video control information; a display outputdevice capable of displaying and outputting the reproduced sub-videoinformation over the reproduced video information; and a control devicefor controlling the display output device, to display-control thesub-video information in the methods and display the sub-videoinformation over the main-video, on the basis of the control informationelement whose presence or effectiveness is shown by the information toindicate presence or absence, or effectiveness or ineffectiveness of thecontrol information element which is included in the reproducedsub-video control information, said computer program product making thecomputer function as at least a part of the first record device, thesecond record device, the third record device, the reproduction device,the display output device and the control device.
 42. A data structureincluding a control signal, said data structure having: videoinformation to indicate a main-video; sub-video information to indicatea sub-video displayable at least partially over the main-video; andsub-video control information including (i) a control informationelement for display control of the sub-video information in a method setin advance and (ii) information to indicate presence or absence, oreffectiveness or ineffectiveness of the control information elements.43. The data structure including a control signal according to claim 42,wherein the sub-video control information includes (i) a plurality oftypes of control information elements for display control of thesub-video information in various methods set in advance and (ii) typeindicating information to indicate presence or absence, or effectivenessor ineffectiveness of the control information elements by each of thetypes.